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Throughout the course of history, engineering and mathematics have developed in 
parallel. All branches of engineering depend on mathematics for their description and 
there has been a steady flow of ideas and problems from engineering that has stimulated 
and sometimes initiated branches of mathematics. Thus it is vital that engineering stu- 
dents receive a thorough grounding in mathematics, with the treatment related to their 
interests and problems. As with the previous editions, this has been the motivation for 
the production of this fourth edition — a companion text to the fourth edition of Modern 
Engineering Mathematics, this being designed to provide a first-level core studies 
course in mathematics for undergraduate programmes in all engineering disciplines. 
Building on the foundations laid in the companion text, this book gives an extensive 
treatment of some of the more advanced areas of mathematics that have applications in 
various fields of engineering, particularly as tools for computer-based system model- 
ling, analysis and design. Feedback, from users of the previous editions, on subject 
content has been highly positive indicating that it is sufficiently broad to provide the 
necessary second-level, or optional, studies for most engineering programmes, where 
in each case a selection of the material may be made. Whilst designed primarily for use 
by engineering students, it is believed that the book is also suitable for use by students 
of applied mathematics and the physical sciences. 

Although the pace of the book is at a somewhat more advanced level than the com- 
panion text, the philosophy of learning by doing is retained with continuing emphasis 
on the development of students’ ability to use mathematics with understanding to solve 
engineering problems. Recognizing the increasing importance of mathematical model- 
ling in engineering practice, many of the worked examples and exercises incorporate 
mathematical models that are designed both to provide relevance and to reinforce the 
role of mathematics in various branches of engineering. In addition, each chapter con- 
tains specific sections on engineering applications, and these form an ideal framework 
for individual, or group, study assignments, thereby helping to reinforce the skills of 
mathematical modelling, which are seen as essential if engineers are to tackle the 
increasingly complex systems they are being called upon to analyse and design. The 
importance of numerical methods in problem solving is also recognized, and its treat- 
ment is integrated with the analytical work throughout the book. 

Much of the feedback from users relates to the role and use of software packages, 
particularly symbolic algebra packages. Without making it an essential requirement the 
authors have attempted to highlight throughout the text situations where the user could 
make effective use of software. This also applies to exercises and, indeed, a limited 
number have been introduced for which the use of such a package is essential. Whilst 
any appropriate piece of software can be used, the authors recommend the use of 
MATLAB and/or MAPLE. In this new edition more copious reference to the use of these 
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two packages is made throughout the text, with commands or codes introduced and 
illustrated. When indicated, students are strongly recommended to use these packages 
to check their solutions to exercises. This is not only to help develop proficiency in their 
use, but also to enable students to appreciate the necessity of having a sound knowledge 
of the underpinning mathematics if such packages are to be used effectively. Throughout 
the book two icons are used: 


e Anopen screen indicates that the use of a software package would be useful 
(e.g. for checking solutions) but not essential. 


e Aclosed screen (e) indicates that the use of a software package is essential or 


highly desirable. 


As indicated earlier, feedback on content from users of previous editions has been 
favourable, and consequently no new chapter has been introduced. However, in 
response to feedback the order of presentation of chapters has been changed, with a 
view to making it more logical and appealing to users. This re-ordering has necessitated 
some redistribution of material both within and across some of the chapters. Another 
new feature is the introduction of the use of colour. It is hoped that this will make the text 
more accessible and student-friendly. Also, in response to feedback individual chapters 
have been reviewed and updated accordingly. The most significant changes are: 


e Chapter | Matrix Analysis: Inclusion of new sections on ‘Singular value decom- 
position’ and ‘Lyapunov stability analysis’. 

e Chapter 5 Laplace transform: Following re-ordering of chapters a more unified 
and extended treatment of transfer functions/transfer matrices for continuous- 
time state-space models has been included. 

e Chapter 6 Z-transforms: Inclusion of a new section on ‘Discretization of 
continuous-time state-space models’. 

e Chapter 8 Fourier transform: Inclusion of a new section on ‘Direct design of 
digital filters and windows’. 

e Chapter 9 Partial differential equations: The treatment of first order equations 
has been extended and a new section on ‘Integral solution’ included. 

e Chapter 10 Optimization: Inclusion of a new section on ‘Least squares’. 


A comprehensive Solutions Manual is available free of charge to lecturers adopting this 
textbook. It will also be available for download via the Web at: www.pearsoned.co.ck/james. 
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1.1 


1 


Introduction 


In this chapter we turn our attention again to matrices, first considered in Chapter 5 
of Modern Engineering Mathematics, and their applications in engineering. At the 
outset of the chapter we review the basic results of matrix algebra and briefly introduce 
vector spaces. 

As the reader will be aware, matrices are arrays of real or complex numbers, and have 
a special, but not exclusive, relationship with systems of linear equations. An (incorrect) 
initial impression often formed by users of mathematics is that mathematicians have 
something of an obsession with these systems and their solution. However, such systems 
occur quite naturally in the process of numerical solution of ordinary differential equa- 
tions used to model everyday engineering processes. In Chapter 9 we shall see that they 
also occur in numerical methods for the solution of partial differential equations, for 
example those modelling the flow of a fluid or the transfer of heat. Systems of linear 
first-order differential equations with constant coefficients are at the core of the state- 
space representation of linear system models. Identification, analysis and indeed design 
of such systems can conveniently be performed in the state-space representation, with 
this form assuming a particular importance in the case of multivariable systems. 

In all these areas it is convenient to use a matrix representation for the systems under 
consideration, since this allows the system model to be manipulated following the rules 
of matrix algebra. A particularly valuable type of manipulation is simplification in some 
sense. Such a simplification process is an example of a system transformation, carried 
out by the process of matrix multiplication. At the heart of many transformations are 
the eigenvalues and eigenvectors of a square matrix. In addition to providing the means 
by which simplifying transformations can be deduced, system eigenvalues provide vital 
information on system stability, fundamental frequencies, speed of decay and long-term 
system behaviour. For this reason, we devote a substantial amount of space to the 
process of their calculation, both by hand and by numerical means when necessary. Our 
treatment of numerical methods is intended to be purely indicative rather than complete, 
because a comprehensive matrix algebra computational tool kit, such as MATLAB, is 
now part of the essential armoury of all serious users of mathematics. 

In addition to developing the use of matrix algebra techniques, we also demonstrate 
the techniques and applications of matrix analysis, focusing on the state-space system model 
widely used in control and systems engineering. Here we encounter the idea of a function 
of a matrix, in particular the matrix exponential, and we see again the role of the 
eigenvalues in its calculation. This edition also includes a section on singular value 
decomposition and the pseudo inverse, together with a brief section on Lyapunov stability 
of linear systems using quadratic forms. 


Review of matrix algebra 


This section contains a summary of the definitions and properties associated with matrices 
and determinants. A full account can be found in chapters of Modern Engineering 
Mathematics or elsewhere. It is assumed that readers, prior to embarking on this chapter, 
have a fairly thorough understanding of the material summarized in this section. 
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Definitions 


(a) An array of real numbers 


Qi Ag 43 Qin 
A- G7, Aq7_— 93 Ayn 
Ami Am2 an3 ene Amn 


is called an m Xn matrix with m rows and n columns. The a, is referred to as the 
i, jth element and denotes the element in the ith row and jth column. If m =n 
then A is called a square matrix of order n. If the matrix has one column or one 
row then it is called a column vector or a row vector respectively. 


(b) Ina square matrix A of order n the diagonal containing the elements a,,, dy, ..., 
a,, 18 called the principal or leading diagonal. The sum of the elements in this 
diagonal is called the trace of A, that is 


n 
trace A = >, aij; 


i=l 


(c) A diagonal matrix is a square matrix that has its only non-zero elements along the 
leading diagonal. A special case of a diagonal matrix is the unit or identity matrix / 
for which a,,=a,=...=a ii 


mn 


(d) A zero or null matrix 0 is a matrix with every element zero. 


(e) The transposed matrix A’ is the matrix A with rows and columns interchanged, 
its i, jth element being a;,. 


(f) A square matrix A is called a symmetric matrix if A’ = A. It is called skew 
symmetric if A’ = —A. 
Basic operations on matrices 


In what follows the matrices A, B and C are assumed to have the i, jth elements a 
and c, respectively. 


iv By 
Equality 


The matrices A and B are equal, that is A = B, if they are of the same order m x n 
and 


l<ism, 1<5jS<n 


Multiplication by a scalar 


If J is a scalar then the matrix AA has elements /a,,. 
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Addition 


We can only add an m x n matrix A to another m x n matrix B and the elements of the 
sum A+ Bare 


a,+b;, 1<i<m, 1s<j<n 


ip? 


Properties of addition 

(i) commutativelaw: A+B=B+A 

(ii) associative law: (A+ B)+ C=A+(B+C) 
(iii) distributive law: A(A+B)=AA+ AB, A scalar 


Matrix multiplication 


If A is an m X p matrix and Ba p x n matrix then we define the product C= AB as the 
m Xn matrix with elements 


Properties of multiplication 


(i) |The commutative law is not satisfied in general; that is, in general AB + BA. 
Order matters and we distinguish between AB and BA by the terminology: 
pre-multiplication of B by A to form AB and post-multiplication of B by A to 
form BA. 


(ii) Associative law: A(BC)=(AB)C 
(iii) If/A is a scalar then 
(AA)B = A(AB) = AAB 
(iv) Distributive law over addition: 
(A+ B)C=AC+ BC 
A(B+ C)=AB+ AC 
Note the importance of maintaining order of multiplication. 


(v) If Ais an m Xn matrix and if f,, and J, are the unit matrices of order m and n 
respectively then 


[,A=Al,=A 


m 


Properties of the transpose 

If A" is the transposed matrix of A then 
G@) (A+B)=A'+B" 

(ii) (A‘)'=A 

(iii) (AB)'= BTAT 
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Determinants 


The determinant of a square n x n matrix A is denoted by det A or |A|. 

If we take a determinant and delete row i and column / then the determinant 
remaining is called the minor M,, of the i, jth element. In general we can take any row 
i (or column) and evaluate an n x n determinant | A | as 


|A|= >. (-1)'" a,Mj 
j=l 


A minor multiplied by the appropriate sign is called the cofactor A, of the i, jth element 
so A, = (-1)” M; and thus 


n 
|A|= Dy aj Aj 
j=l 


Some useful properties 
(i) |A*|=IAl 
(ii) |AB|=|A||B| 


(iii) A square matrix A is said to be non-singular if | A| #0 and singular if | A| = 0. 


Adjoint and inverse matrices 


Adjoint matrix 


The adjoint of a square matrix A is the transpose of the matrix of cofactors, so for a 
3 x 3 matrix A 


T 
Ay Ay Aj 

adj A= | A,, Ax Ap; 
Ax, Ax A33 


Properties 

(i) A(adj A) =|A|! 

(ii) |adj A|=|A|"", n being the order of A 
(iii) adj (AB) = (adj B)(adj A) 


Inverse matrix 


Given a square matrix A if we can construct a square matrix B such that 
BA= AB=! 


then we call B the inverse of A and write it as A". 
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Properties 

(i) If Ais non-singular then |A| #0 and A" = (adj A)/|A|. 
(ii) If A is singular then |A|=0 and A" does not exist. 
(iii) (ABY' = BOA". 


All the basic matrix operations may be implemented in MATLAB and MAPLE 
using simple commands. In MATLAB a matrix is entered as an array, with row 
elements separated by spaces (or commas) and each row of elements separated by a 
semicolon(;), or the return key to go to a new line. Thus, for example, 

Mell 2 Se @& 0 Se 7 a 2] 
gives 


Having specified the two matrices A and B the operations of addition, subtraction 

and multiplication are implemented using respectively the commands 
C=A+B, C€=A-B, C-A*B 

The trace of the matrix A is determined by the command trace (A), and its 
determinant by det (A). 

Multiplication of a matrix A by a scalar is carried out using the command *, while 
raising A to a given power is carried out using the command ~* . Thus, for example, 
3A’ is determined using the command c=3*A*2. 

The transpose of a real matrix A is determined using the apostrophe ’ key; that 
is C=A’ (to accommodate complex matrices the command c=A.’ should be used). 
The inverse of A is determined by C=inv(A). 

For matrices involving algebraic quantities, or when exact arithmetic is desirable 
use of the Symbolic Math Toolbox is required; in which matrices must be expressed 
in symbolic form using the sym command. The command A=sym(A) generates the 
symbolic form of A. For example, for the matrix 


DM 37 WS 
A=/12 05 33 
3 lil @ 


the commands 
Bel. Se2 Oo6¢ 1.2 O25 2.39 Sos tei Ole 
A=sym (A) 
generate 
he 
[2i7lO, L6/S, 3/5] 
l6/a, i724, Baio] 
PA'S, LALA), (0)]| 


Symbolic manipulation can also be undertaken in MATLAB using the MuPAD 
version of Symbolic Math Toolbox. 
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There are several ways of setting up arrays in MAPLE; the easiest is to use the 
linear algebra package LinearAlgebra so, for example, the commands: 


with (LinearAlgebra) : 
Mg=wisicmsss (Tp 2,3 nlc O, Slo lioGr2) 1) e 


return 
ee es 
A=|]4 0 5 
Wo & 2 


with the command 


iggaweciitore (LZ, 3'q IL] })s 


returning 
” 
Igy = || 3 
al 


Having specified two matrices ‘A and B’ addition and subtraction are implemented 
using the commands: 


€:=A+B; and C:=A—B; 


Multiplication of a matrix A by a scalar & is implemented using the command k*a; 
so, for example, (2A + 3B) is implemented by 


2*A+3*B; 
The product AB of two matrices is implemented by either of the following two 
commands: 

Ate © Wille sell (A, 1) 2 


(Note: A*B will not work) 
The transpose, trace, determinant, adjoint and inverse of a matrix A are returned 
using, respectively, the commands: 


Transpose (A) ; 


Trace (A); 
Determinant (A) ; 
Adjoint (A) ; 
MatrixInverse (A); 


1.2.5 Linear equations 


In this section we reiterate some definitive statements about the solution of the system 
of simultaneous linear equations 


Qy)X, + ap +... +4),X, = d, 


Og Xp Opto Fk. Gy, x; = D5 


AyX] + An2X2 Feet QinXn = b, 
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or, in matrix notation, 


Gy, Ayn wee Ain | | XY by 
Gy, An7— +++ Aan | | X02] _ b, 
ant an2 oa 28 Qin xX; b, 
that is, 
Ax=b (1.1) 


where A is the matrix of coefficients and x is the vector of unknowns. If b = 0 the 
equations are called homogeneous, while if b # 0 they are called nonhomogeneous (or 
inhomogeneous). Considering individual cases: 


Case (i) 


If b #0 and |A| # 0 then we have a unique solution x = Ab. 


Case (ii) 


If b= 0 and |A| #0 we have the trivial solution x = 0. 


Case (iii) 
If b #0 and|A|=0 then we have two possibilities: either the equations are inconsistent 
and we have no solution or we have infinitely many solutions. 


Case (iv) 
If b= 0 and |A| = 0 then we have infinitely many solutions. 
Case (iv) is one of the most important, since from it we can deduce the important 


result that the homogeneous equation Ax = 0 has a non-trivial solution if and only 
if |A|=0. 


Provided that a solution to (1.1) exists it may be determined in MATLAB using the 
command x=A\b. For example, the system of simultaneous equations 
x+y+z=6, x+2y+3z=14, x+4y+9z=36 


may be written in the matrix form 


oi YP ae 6 
1 2 3]lyl/=|14 
1 4 94JLz 36 

A EM b 


Entering A and b and using the command x = A\b provides the answer x= 1, y= 2, z=3. 
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In MAPLE the commands 


with (LinearAlgebra) : 
soln:=LinearSolve(A,b); 


will solve the set of linear equations Ax = b for the unknown x when A, b given. 
Thus for the above set of equations the commands 


with (LinearAlgebra) : 

Ag=Metceis<e (i ,L,1),lL,2,31, 12,424,811) 3 
logeWeetor (6,14, 361) 7 
x:=LinearSolve(A,b); 


return 


Rank of a matrix 


The most commonly used definition of the rank, rank A, of a matrix A is that it is the order 
of the largest square submatrix of A with a non-zero determinant, a square submatrix 
being formed by deleting rows and columns to form a square matrix. Unfortunately it 
is not always easy to compute the rank using this definition and an alternative definition, 
which provides a constructive approach to calculating the rank, is often adopted. First, 
using elementary row operations, the matrix A is reduced to echelon form 


* 


* 


Te 


in which all the entries below the line are zero, and the leading element, marked *, in 
each row above the line is non-zero. The number of non-zero rows in the echelon form 
is equal to rank A. 

When considering the solution of equations (1.1) we saw that provided the determinant 
of the matrix A was not zero we could obtain explicit solutions in terms of the inverse matrix. 
However, when we looked at cases with zero determinant the results were much less clear. 
The idea of the rank of a matrix helps to make these results more precise. Defining the 
augmented matrix (A : 5) for (1.1) as the matrix A with the column b added to it then 
we can state the results of cases (111) and (iv) of Section 1.2.5 more clearly as follows: 


If A and (A: b) have different rank then we have no solution to (1.1). If the two 


matrices have the same rank then a solution exists, and furthermore the solution 
will contain a number of free parameters equal to (m — rank A). 
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(2) In MATLAB the rank of the matrix A is generated using the command rank (A). 
For example, if 

—1 

A=| 0 

=1 


a) 
So -= & 


the commands 


emi 2 Be © @ ie =1 2 Olle 
rank (A) 


generate 
ans=2 


In MAPLE the command is also rank (A) . 


Vector spaces 


Vectors and matrices form part of a more extensive formal structure called a vector space. 
The theory of vector spaces underpins many modern approaches to numerical methods 
and the approximate solution of many of the equations that arise in engineering analysis. 
In this section we shall, very briefly, introduce some of the basic ideas of vector spaces 
necessary for later work in this chapter. 


Definition 

A real vector space / is a set of objects called vectors together with rules for addition 
and multiplication by real numbers. For any three vectors a, b and c in V and any real 
numbers a and f the sum a + b and the product aa also belong to V and satisfy the 
following axioms: 


(a) at+b=bt+a 

(0) @ar (see) = (@ ae D) ae 

(c) there exists a zero vector 0 such that 
a+0=a 

(d) for each a in V there is an element —a in V such that 
a+(-a)=0 

(ec) OA(a+ b)=aa+ ab 

(f) (a+ B)a=aa+t Ba 

(g) (@B)a = o(Ba) 

(h) la=a 
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Example 1.1 


Solution 


1.3 VECTOR SPACES 11 


It is clear that the real numbers form a vector space. The properties given are also 
satisfied by vectors and by m X n matrices so vectors and matrices also form vector 
spaces. The space of all quadratics a + bx + cx’ forms a vector space, as can be estab- 
lished by checking the axioms, (a)—(h). Many other common sets of objects also form 
vector spaces. If we can obtain useful information from the general structure then this 
will be of considerable use in specific cases. 


Linear independence 


The idea of linear dependence is a general one for any vector space. The vector x is said 
to be linearly dependent on x,, x,, ..., x,, if it can be written as 


m 
X= OX, + OX, +... + OX y 
for some scalars a, ..., &,,. The set of vectors y,, y>,..., y,, 1S said to be linearly 
independent if and only if 
By, + Boy, +... + BV = 0 


implies that 8, = B,=...=8,,=0. 
Let us now take a linearly independent set of vectors x,, x5, ...,X,, in V and con- 
struct a set consisting of all vectors of the form 


X= A,X, + A,X, +...+ ,,X,, 


We shall call this set S(x,, x5, ...,,,). It is clearly a vector space, since all the axioms 
are satisfied. 


Show that 
1 0 
e,=|/0| and e,=/1 
0 0 


form a linearly independent set and describe S(e,, e,) geometrically. 


We have that 
a 
0 = ae, + Be, = | B 
0 


is only satisfied if «= B= 0, and hence e, and e, are linearly independent. 


a 
S(@;, @)) is the set of all vectors of the form | B|, which is just the (x), x2) 


0 


plane and is a subset of the three-dimensional Euclidean space. 
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142 


If we can find a set B of linearly independent vectors x,, x,,..., x, in V such that 
S(X1,X%,..-,XJ)=V 


then B is called a basis of the vector space V. Such a basis forms a crucial part of the 
theory, since every vector x in V can be written uniquely as 


X= OX, + A,X, +...+ OX, 


The definition of B implies that x must take this form. To establish uniqueness, let us 
assume that we can also write x as 


x= Bx, + Box.+...+ Bx, 
Then, on subtracting, 


0=(@,-— Bx, +-..+(a,— Bx, 


and since x,,..., xX, are linearly independent, the only solution is a, = B,, @ = B.,...; 
hence the two expressions for x are the same. 

It can also be shown that any other basis for V must also contain n vectors and that 
any n + | vectors must be linearly dependent. Such a vector space is said to have 
dimension 7 (or infinite dimension if no finite n can be found). In a three-dimensional 
Euclidean space 


1 0 
€é,=/0]/, e=]/1]/, e= 
0 0 1 


1 1 1 
d,= 0}, d,= 1 ’ d,= 1 
0 0 1 


is also a perfectly good basis. While the basis can change, the number of vectors in the 
basis, three in this case, is an intrinsic property of the vector space. If we consider the 
vector space of quadratics then the sets of functions {1, x, x°} and {1, x — 1, x(x — 1)} 
are both bases for the space, since every quadratic can be written as a + bx + cx* or as 
A+ B(x — 1) + Cx(x — 1). We note that this space is three-dimensional. 


Transformations between bases 


Since any basis of a particular space contains the same number of vectors, we can look 
at transformations from one basis to another. We shall consider a three-dimensional 
space, but the results are equally valid in any number of dimensions. Let e,, e5, e; and 
e;, €5, e; be two bases of a space. From the definition of a basis, the vectors e/, e; and e; 
can be written in terms of e,, e, and e; as 


, 
€) = Ae; + Ay @y + 4313 
C5 = Ay2€) + Ane, + Aye; (1.2) 


, 
€3 = A383 + A738) + 3323 
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Taking a typical vector x in V, which can be written both as 
X= Xe; + X,e) + X3€; (1.3) 
and as 
X= xe, + .x5e5 + GE; 
we can use the transformation (1.2) to give 
=" , , 
X= X7(Ae1 + Ay ey + y1@3) + .X3(A12e1 + Aney + Ayy€3) + X3(A3€; + Ay3€) + Ay3€3) 
, , , , , ie - , , 
= (XA + XQAyy + X4Ay3)E, + (XjAqy + XGAyq + XZAq3)Ey + (X13, + XAgy + 433 )es 
On comparing with (1.3) we see that 
, /, /, 
Xy = AX + Ay) + 1 3%3 
= , , , 
Xq = Ay 1X1 + AgXy + A73%3 
= , , , 
X3 = Az 1X1 + Az2Xy + A33%3 
or 
x=Ax 


Thus changing from one basis to another is equivalent to transforming the coordinates 
by multiplication by a matrix, and we thus have another interpretation of matrices. 
Successive transformations to a third basis will just give x’ = Bx”, and hence the 
composite transformation is x = (AB)x” and is obtained through the standard matrix 
tules. 

For convenience of working it is usual to take mutually orthogonal vectors as a 
basis, so that e;e,= 5, and e/'e; = 6,, where 6, is the Kronecker delta 


i> 


Using (1.2) and multiplying out these orthogonality relations, we have 


*T > T T 
eas Y Apes %. ae, = +>. AyiApjCk Cp = >. ApjApjOky = 3 Air 
Dp P kp k 


k k 


Hence 


ey Agia = 6 


k 
or in matrix form 
A'A=! 


It should be noted that such a matrix A with A’! = A’ is called an orthogonal 
matrix. 
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1.3.3 Exercises 


1 Which of the following sets form a basis for a 
three-dimensional Euclidean space? 


1] ]1 

(a) | 0, 
ry 

(c) |O}, | 1 
L 4 | 9] 


N 


Under this, how does the vector 
X =x), + X,e, + x3e; transform and what 
is the geometrical interpretation? What 


1 1 3 lines transform into scalar multiples of 
(b) |O}, | 2], }2 themselves? 
1) |3} {5 


3. Show that the set of all cubic polynomials 
forms a vector space. Which of the following 
sets of functions are bases of that space? 


(a) {1, x, 2°, x7} 
(b) {l1-x,1+x,1-»,1+x} 


(c) {1—x, 1 +x, 2(1 —x), 2(1 +x)} 


0 0 
e=l1), e=/0 (d) {x(1 -x), x1 +x), 1-3, 1 +37} 
0 1 (e) {1 + 2x, 2x + 3x2, 3x2 + 4x3, 4x3 + 1} 


find the transformation that takes these to the vectors 


er = 


1.4 


1 
1 
2|") 
0 


Describe the vector space 


1 0 
es = + -1|, e =|0 S(x + 2x7, 2x — 3x°, x +x°) 
\ 
0 1 What is its dimension? 


The eigenvalue problem 


A problem that leads to a concept of crucial importance in many branches of math- 
ematics and its applications is that of seeking non-trivial solutions x # 0 to the matrix 
equation 


Ax=Ax 


This is referred to as the eigenvalue problem; values of the scalar 2 for which non- 
trivial solutions exist are called eigenvalues and the corresponding solutions x # 0 are 
called the eigenvectors. Such problems arise naturally in many branches of engineering. 
For example, in vibrations the eigenvalues and eigenvectors describe the frequency and 
mode of vibration respectively, while in mechanics they represent principal stresses 
and the principal axes of stress in bodies subjected to external forces. In Section 1.11, 
and later in Section 5.7.1, we shall see that eigenvalues also play an important role in 
the stability analysis of dynamical systems. 

For continuity some of the introductory material on eigenvalues and eigenvectors, 
contained in Chapter 5 of Modern Engineering Mathematics, is first revisited. 
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Solution 
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The characteristic equation 


The set of simultaneous equations 


Ax = Ax (1.4) 
where A is an n X n matrix andx=[x, x, ... x,]' is ann x 1 column vector can 
be written in the form 

(Al- A)x = 0 (1.5) 


where / is the identity matrix. The matrix equation (1.5) represents simply a set of 
homogeneous equations, and we know that a non-trivial solution exists if 


c(A) = |Al- A] =0 (1.6) 


Here c(A) is the expansion of the determinant and is a polynomial of degree n in A, 
called the characteristic polynomial of A. Thus 


CA) = +6, 4146.8? +...4 0A +6 
n-l n—2 1 0 


and the equation c(A) = 0 is called the characteristic equation of A. We note that this 
equation can be obtained just as well by evaluating |A — A/| = 0; however, the form 
(1.6) is preferred for the definition of the characteristic equation, since the coefficient 
of 2’ is then always +1. 

In many areas of engineering, particularly in those involving vibration or the control 
of processes, the determination of those values of A for which (1.5) has a non-trivial 
solution (that is, a solution for which x # 0) is of vital importance. These values of 
A are precisely the values that satisfy the characteristic equation, and are called the 
eigenvalues of A. 


Find the characteristic equation for the matrix 


1 1 -2 
A=|-1 2 1 
0 1 -l 


By (1.6), the characteristic equation for A is the cubic equation 
A-1 -1 2 
c(A) =| 1 A-2 -1/=0 
0 -1 A+1 
Expanding the determinant along the first column gives 
A-2 -1 -1 2 
A) =(A-1 - 

a ) -1 ee F A+] 

=(A-)I[A-2A+ )-1)-2-A+)] 
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Thus 
(A) = # - 28 —-A4+2=0 


is the required characteristic equation. 


For matrices of large order, determining the characteristic polynomial by direct 
expansion of |A/— A| is unsatisfactory in view of the large number of terms involved 
in the determinant expansion. Alternative procedures are available to reduce the amount 
of calculation, and that due to Faddeev may be stated as follows. 


The method of Faddeev 


If the characteristic polynomial of an 7 x n matrix A is written as 


x pA Seegite Pyar Pn 


then the coefficients p,, p,,..., p, can be computed using 
1 _ 
p,=-traceA, (r=1,2,...,n) 
‘a 


where 


r 


_fA (r=1) 
~ |AB,, (r=2,3,...,7) 


and 
B.=A,-—p,l, where /is the n x n identity matrix 
The calculations may be checked using the result that 


B,=A,-p,! must be the zero matrix 


Example 1.3 —_ Using the method of Faddeev, obtain the characteristic equation of the matrix A of 
Example 1.2. 


Solution 1 1 -2 


Let the characteristic equation be 


c(A) = x —p,X — poh — p; 
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Then, following the procedure described above, 


Pp, =traceA=(1+2-1)=2 


af 1. =o 
B,=A-2!l=|-1 0 1 
a es 

a sat 
A,=AB,=|-1 0 1 
of 21 a 


P2 = strace A, = $(-2+0+4)=1 


4 <i. 
B=An-FS\ =). 7 
4 <1 3 
~2 0| 
A,=AB,=| 0 2 0 
0 -2| 


p3= ftraceA, = 1(-2-2-2)=-2 
Then, the characteristic polynomial of A is 
c(A) = #- 27 -2+2 


in agreement with the result of Example 1.2. In this case, however, a check may be 
carried out on the computation, since 


B,=A,+2!=0 


as required. 


Eigenvalues and eigenvectors 


The roots of the characteristic equation (1.6) are called the eigenvalues of the matrix A 
(the terms latent roots, proper roots and characteristic roots are also sometimes used). 
By the Fundamental Theorem of Algebra, a polynomial equation of degree n has 
exactly n roots, so that the matrix A has exactly n eigenvalues 1,,i=1,2,...,n. These 
eigenvalues may be real or complex, and not necessarily distinct. Corresponding to each 
eigenvalue /,, there is a non-zero solution x = e, of (1.5); e, is called the eigenvector of 
A corresponding to the eigenvalue /,. (Again the terms latent vector, proper vector and 
characteristic vector are sometimes seen, but are generally obsolete.) We note that if 
x = e, satisfies (1.5) then any scalar multiple B,e, of e, also satisfies (1.5), so that the 
eigenvector e, may only be determined to within a scalar multiple. 
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Example 1.4 Determine the eigenvalues and eigenvectors for the matrix A of Example 1.2. 


Solution ti tf = 
A=|-1 2 1 
0 —l 


The eigenvalues A, of A satisfy the characteristic equation c(A) = 0, and this has been 
obtained in Examples 1.2 and 1.3 as the cubic 


KB -2H -2A+2=0 


which can be solved to obtain the eigenvalues A,, 2, and A. 

Alternatively, it may be possible, using the determinant form |A/-— Al, or indeed (as 
we often do when seeking the eigenvalues) the form |A — Al|, by carrying out suitable 
row and/or column operations to factorize the determinant. 

In this case 


t=% 4 =) 
|A-Al|=|-1 2-A 1 
0 1 =e 


and adding column | to column 3 gives 
1-A 1 -1-aA 1-a 1 1 
-l1 2-A 0 =-(1+A)| -1 2-A 0 
0 1 -1-A 0 1 
Subtracting row 3 from row | gives 


=n 7 % 
Al49)) =k 2H DS alee HH 
0 1 1 


Setting |A — Al | = 0 gives the eigenvalues as A, = 2, A, = 1 and A, =—1. The order in 
which they are written is arbitrary, but for consistency we shall adopt the convention of 
taking A,,A,,..., A, in decreasing order. 

Having obtained the eigenvalues /, (i = 1, 2, 3), the corresponding eigenvectors e; 
are obtained by solving the appropriate homogeneous equations 


(A-ANe, = 0 (1.7) 


When i= 1, 4;= A, = 2 and (1.7) is 


-l 1 —2 ei 
-1 0 I|le,/=0 
0 —3]} ey 
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that is, 
—€,,;+ e@,)—2e,,=0 
—e,,+0e,+ e,=0 
0e,,;+ ey —3e,=0 


leading to the solution 


where f, is an arbitrary non-zero scalar. Thus the eigenvector e, corresponding to the 
eigenvalue A, = 2 is 


e,=B [1 3 1" 


As a check, we can compute 


1 1 -2})1 2 1 
Ae, = B, -l 2 1 3} = B, 6| = 2B, oa Aye; 
0 1 -ly}1 2 1 


and thus conclude that our calculation was correct. 
When i = 2, 4,=A,= 1 and we have to solve 


0 1 -2])e, 
11. I|len/=0 
0 1 —2 C53 


that is, 
0e,; + €y) — 2), = 0 
—€5, Fey + €3=0 
0e,; + €y) — 2€), = 0 


leading to the solution 


Cn, _ 20 _ 23 
3 ye 


-3 2 


where 3, is an arbitrary scalar. Thus the eigenvector e, corresponding to the eigenvalue 
A,=1is 


e,=B,[3 2 1" 


Again a check could be made by computing Ae,. 
Finally, when i = 3, A; =A, =—1 and we obtain from (1.7) 


2 1 -2)) e3, 
-1 3 1||e.) =0 
0 1 0] | e353 
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that is, 
2€3,+ €3) — 2e3; = 0 
—€3, + 3e3 + €33=0 
0e;;+ e3,+ Oe, =0 


and hence 


-1 0 


C31 _ 30 33 B 
—_ = = — = 3 
-l 


Here again f; is an arbitrary scalar, and the eigenvector e; corresponding to the eigen- 
value A, is 


e,=B;[1 0 iy? 


The calculation can be checked as before. Thus we have found that the eigenvalues of 
the matrix A are 2, 1 and —1, with corresponding eigenvectors 


Bll 3 if, 6B. 2 if and 61 0 1) 


respectively. 


Since in Example 1.4 the f,, i = 1, 2, 3, are arbitrary, it follows that there are an 
infinite number of eigenvectors, scalar multiples of each other, corresponding to each 
eigenvalue. Sometimes it is convenient to scale the eigenvectors according to some 
convention. A convention frequently adopted is to normalize the eigenvectors so that 
they are uniquely determined up to a scale factor of +1. The normalized form of an 


eigenvectore=[e, e, ... e,]' is denoted by é and is given by 
ga. 
le| 
where 
i 2 2 2 
jJel|= (ej +er+ ... +e,) 


For example, for the matrix A of Example 1.4, the normalized forms of the eigenvectors 
are 


B=(Wli Sil: iiiy, é,=([3//14 2//14 /y14f" 
and 
é,=[1/j2 0 1/2)" 


However, throughout the text, unless otherwise stated, the eigenvectors will always 
be presented in their ‘simplest’ form, so that for the matrix of Example 1.4 we take 
B, = B, = B; = 1 and write 


e=[1 3 1], e=[8 2 1]' and e,=[1 0 1]' 
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For an X n matrix A the MATLAB command p=poly (A) generates an n+ 1 ele- 
ment row vector whose elements are the coefficients of the characteristic polynomial 
of A, the coefficients being ordered in descending powers. The eigenvalues of A 
are the roots of the polynomial and are generated using the command roots (p). 
The command 


[M,S]=e1ig (A) 


generates the normalized eigenvectors of A as the columns of the matrix M and its 
corresponding eigenvalues as the diagonal elements of the diagonal matrix S 
(Mand S are called respectively the modal and spectral matrices of A and we shall 
return to discuss them in more detail in Section 1.6.1). In the absence of the left- 
hand arguments, the command eig(A) by itself simply generates the eigenvalues 
of A. 

For the matrix A of Example 1.4 the commands 


Ae I =29 = 2 ig © 1 =L)s 
[M, S]=eig (A) 
generate the output 


OQ. SOLS =O Sits O. 707A 
M=a0) 04S =O psal's 0.0000 
@ SOLS 0), 2673) 0.7071 


2.0000 0 0) 
SU 1.0000 0 
0 0 qb, ONG 


These concur with our calculated answers, with B, = 0.3015, B, = —0.2673 and 
(6, =O. 007. 

Using the Symbolic Math Toolbox in MATLAB we saw earlier that the matrix A 
may be converted from numeric into symbolic form using the command A=sym (A). 
Then its symbolic eigenvalues and eigenvectors are generated using the sequence of 
commands 


A=[l 1 =27 =1 2 1; 0 1 —-1]; 
A=sym(A); 
IM, S]|=e1g (A) 

as 


MES, ik, i] 


[Sy 3, Wl) 
[iy Ly 2) 
S=[1, 0, 0] 
[0, 2, 0] 
iO, @_ =a) 


In MAPLE the command Eigenvalues (A); returns a vector of eigenvalues. The 
command Eigenvectors (A) returns both a vector of eigenvalues as before and 
a matrix containing the eigenvalues, so that the ith column is an eigenvector 
corresponding to the eigenvalue in the ith entry of the preceding vector. Thus the 
commands: 
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Example 1.5 


Solution 


with (LinearAlgebra) , 
Ag=Wenezabe (Ll, d,=21, l=l,2, Lp lO,1,=1) 1): 
Eigenvalues (A) ; 


return 


L-1 


and the command 


Eigenvectors (A) ; 


returns 
, 2 i 3 
-l 3 @ 2 
L oil I tL il 


Find the eigenvalues and eigenvectors of 


A= cos@ —sin@ 
sin@  cos@ 


Now 


Ja—al=]"" one sin 0 | 
—sin@ A — cos@ 
= # — 2A cos@+ cos’@+ sin’@= H — 2A cosO+ 1 

So the eigenvalues are the roots of 

# —2Acosd+1=0 
that is, 

A=cos@+ jsind 
Solving for the eigenvectors as in Example 1.4, we obtain 


e,=[1 -jJ' and e=[1 jJ 


In Example 1.5 we see that eigenvalues can be complex numbers, and that the eigen- 
vectors may have complex components. This situation arises when the characteristic 
equation has complex (conjugate) roots. 
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1.4.3 Exercises 


Check your answers using MATLAB or MAPLE whenever possible. 


characteristic polynomials of the matrices 


3 2 1 
(a)}4 5 -1 
23 4 


(a) I 
11 


1.4.4 


Using the method of Faddeev, obtain the fr 10 —4] . 1412 
©)| 05 4 (| 0 2 2 
2 -1 1 2 |-4 | I-] 1 3 
0 1 1 0 
(b) - ; - 
=f 11d 5 0 6 1 -1l 0O 
1 11.0 (e)/0 11 6 (f)) 1 2 
. : : 6 6 -2 -2 -1 
Find the eigenvalues and corresponding o = = 
eigenvectors of the matrices 2 7 7 
4 1 1 1 -4 -2 
1 2 ()/ 2 5 4 (h) | 0 1 
(b) : 
3 2 |-1 -1 0] it 2 @ 
Repeated eigenvalues 
In the examples considered so far the eigenvalues 1; (i= 1, 2,.. . ) of the matrix A have 


Example 1.6 


Solution 


been distinct, and in such cases the corresponding eigenvectors can be found and are 
linearly independent. The matrix A is then said to have a full set of linearly independent 
eigenvectors. It is clear that the roots of the characteristic polynomial c(A) may not all 
be distinct; and when c(A) has p  n distinct roots, c(A) may be factorized as 


cCAHWeday Aaa) andedy” 


indicating that the root A=A,,i=1,2,...,p, isa root of order m,, where the integer m; 
is called the algebraic multiplicity of the eigenvalue 1. Clearly m, + m)+...+m, =n. 
When a matrix A has repeated eigenvalues, the question arises as to whether it is 
possible to obtain a full set of linearly independent eigenvectors for A. We first consider 
two examples to illustrate the situation. 


Determine the eigenvalues and corresponding eigenvectors of the matrix 


3-3 2 
A=|-1 5 -2 
-1 3. (0 


We find the eigenvalues from 


3-A -3 2 
-1 5-A -2/=0 
-1 3. -A 


asA,=4,4,=A,=2. 
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EJ 


The eigenvectors are obtained from 
(A— Ale, =0 (1.8) 
and when A = A, = 4, we obtain from (1.8) 
e=[1 -1 -1]° 
When A= A, =A, = 2, (1.8) becomes 


-1 3-2} | €o3 
so that the corresponding eigenvector is obtained from the single equation 
€y, — 3), + 2e,, = 0 (1.9) 


Clearly we are free to choose any two of the components e,,, e,, or @,, at will, with the 
remaining one determined by (1.9). Suppose we set e,, = oand e,, = f; then (1.9) means 
that e,, = 3a — 28, and thus 


3 —2 
e,=Ba-28B a Bl'=al1|+B) 0 (1.10) 
0 1 


Now A = 2 is an eigenvalue of multiplicity 2, and we seek, if possible, two linearly 
independent eigenvectors defined by (1.10). Setting ~= 1 and B= 0 yields 


a=) 1: 0; 
and setting @ = 0 and B = 1 gives a second vector 
e,=[-2 0 1]' 


These two vectors are linearly independent and of the form defined by (1.10), and it is 
clear that many other choices are possible. However, any other choices of the form (1.10) 
will be linear combinations of e, and e; as chosen above. For example,e=[1 1 1] 
satisfies (1.10), but e = e, + es. 

In this example, although there was a repeated eigenvalue of algebraic multiplicity 2, 
it was possible to construct two linearly independent eigenvectors corresponding to this 
eigenvalue. Thus the matrix A has three and only three linearly independent eigenvectors. 


The MATLAB commands 
Meals =3 2p =i & =f il 3 Olle 
[M,S]=eig (A) 

generate 


OsS0714 =O.5774 =O. 75113 
M=-0.5774 -0.5774 OQ. LSS 
=V,a7 74 =0.57 1/4 ORGsion! 


4.0000 0) 0 
S= 2.0000 0 
0 0 2.0000 
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Clearly the first column of M (corresponding to the eigenvalue A, = 4) is a scalar 
multiple of e, The second and third columns of M (corresponding to the repeated 
eigenvalue A, = A, = 2) are not scalar multiples of e, and e,. However, both satisfy 
(1.10) and are equally acceptable as a pair of linearly independent eigenvectors 
corresponding to the repeated eigenvalue. It is left as an exercise to show that both 
are linear combinations of e, and e,. 

Check that in symbolic form the commands 


A=sym(A); 
[M, S]=e1ig (A) 
generate 

Ms, 3, =2] 
lt, 2, @] 
(it, @, ay 

s=[4, 0, 0] 
IM, 2, Wl 
I@, @, 2] 


In MAPLE the command Eigenvectors (A); produces corresponding results. 
Thus the commands 

with (LinearAlgebra) : 

AgaMieieieiss (S,=—3,21 5 l=leS,=21,l=1,3,011) 5 
Eigenvectors (A); 


return 


Determine the eigenvalues and corresponding eigenvectors for the matrix 


1 2 2 
A=; 0 2 1 
=i. 2.. 2 


Solving |A — Al| = 0 gives the eigenvalues as 1, = A, = 2, A; = 1. The eigenvector 
corresponding to the non-repeated or simple eigenvalue 1, = | is easily found as 


eg=[1 1 17 
When A=/, =A, = 2, the corresponding eigenvector is given by 
(A- 2I)e, =0 


that is, as the solution of 


—e,, + 2e,, + 2e,,=0 (i) 
@3=0 (ii) 
—e,, + 2e, =0 (iii) 
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Example 1.8 


Solution 


From (ii) we have e,, = 0, and from (1) and (ii) it follows that e,, = 2e,,. We deduce 
that there is only one linearly independent eigenvector corresponding to the repeated 
eigenvalue A = 2, namely 


e,=[2 1 oy 


and in this case the matrix A does not possess a full set of linearly independent 
eigenvectors. 


We see from Examples 1.6 and 1.7 that if an n x n matrix A has repeated eigen- 
values then a full set of n linearly independent eigenvectors may or may not exist. 
The number of linearly independent eigenvectors associated with a repeated eigen- 
value A, of algebraic multiplicity m, is given by the nullity g, of the matrix A - A,f, 
where 


g,=n—rank (A-A,J), with 1<q,;<m,; (1.11) 
q; is sometimes referred to as the degeneracy of the matrix A — /,/ or the geometric 


multiplicity of the eigenvalue A,, since it determines the dimension of the space 
spanned by the corresponding eigenvector(s) e;. 


Confirm the findings of Examples 1.6 and 1.7 concerning the number of linearly 
independent eigenvectors found. 


In Example 1.6, we had an eigenvalue 1, = 2 of algebraic multiplicity 2. Correspondingly, 


32, =3 2 1 -3 2 
A-A,l= 1 5-2 -2)=|-1 3-2 
-1 3 2 -1 3-2 


and performing the row operation of adding row | to rows 2 and 3 yields 


—3. 2 


Adding 3 times column | to column 2 followed by subtracting 2 times column | from 
column 3 gives finally 


1 0 0 
0 0 0 
0 0 0 


indicating a rank of |. Then from (1.11) the nullity g, = 3 — 1 = 2, confirming that 
corresponding to the eigenvalue A = 2 there are two linearly independent eigenvectors, 
as found in Example 1.6. 
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In Example 1.7 we again had a repeated eigenvalue A, = 2 of algebraic multiplicity 2. 
Then 


1-2 2 2 -1 2 2 
A-2l= 0 2-2 1 j;=}| 0 0 1 
-1 2 ia) -1 2 0 


Performing row and column operations as before produces the matrix 


this time indicating a rank of 2. From (1.11) the nullity g, = 3 — 2 = 1, confirming that 
there is one and only one linearly independent eigenvector associated with this eigen- 
value, as found in Example 1.7. 


1.4.5 Exercises 


Check your answers using MATLAB or MAPLE whenever possible. 


Obtain the eigenvalues and corresponding 


using the concept of rank, determine how 


eigenvectors of the matrices many linearly independent eigenvectors 
7 a correspond to this value of A. Determine a 
ae 0 <8 2 corresponding set of linearly independent 
(a) |1 3 1 (b) |-1 1 2 eigenvectors. 
Lik ae l= =1 2 
7 = 9 Given that 2 = 1 is a twice-repeated eigenvalue 
- pe 6 ee ee of the matrix 
() ; 1 3 2 (d) |}3 0 -2 
(-l -5 -2 |6 -2 -3 -1 
A=|-1 0 1 
Given that A = 1 is a three-times repeated a 2 > 
eigenvalue of the matrix 
3 7 -5 how many linearly independent eigenvectors 
correspond to this value of A? Determine a 
A=] 2 4 3 : : p 
corresponding set of linearly independent 
1 2 2 eigenvectors. 
1.4.6 Some useful properties of eigenvalues 


The following basic properties of the eigenvalues 4,, 2,,..., A, of ann Xn matrix A 
are sometimes useful. The results are readily proved either from the definition of eigen- 
values as the values of A satisfying (1.4), or by comparison of corresponding charac- 
teristic polynomials (1.6). Consequently, the proofs are left to Exercise 10. 
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Property 1.1 


The sum of the eigenvalues of A is 


n n 
y A, = trace A = Sy ar 
il fil 


Property 1.2 


The product of the eigenvalues of A is 


T+ det A 
ill 


where detA denotes the determinant of the matrix A. 


Property 1.3 


The eigenvalues of the inverse matrix A™', provided it exists, are 


Property 1.4 


The eigenvalues of the transposed matrix A’ are 
A; A 29 OD ot) A n 


as for the matrix A. 


Property 1.5 


If & is a scalar then the eigenvalues of kA are 


JA, Ag) use, 


n 


Property 1.6 


If & is a scalar and / the n x n identity (unit) matrix then the eigenvalues of A + kl 
are respectively 


Ath A,th ..., A,tk 


n 
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Example 1.9 


Solution 
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Property 1.7 


If k is a positive integer then the eigenvalues of A‘ are 


k k k 
MW A>, Soret) A; 


Symmetric matrices 


A square matrix A is said to be symmetric if A' = A. Such matrices form an important 
class and arise in a variety of practical situations. Two important results concerning the 
eigenvalues and eigenvectors of such matrices are 


(a) the eigenvalues of a real symmetric matrix are real; 


(b) for an n X n real symmetric matrix it is always possible to find 7 linearly 
independent eigenvectors e,, e,,..., e, that are mutually orthogonal so 
that eje, = 0 for i #/. 

If the orthogonal eigenvectors of a symmetric matrix are normalized as 
1, Ga, 2+. 5 &, 

then the inner (scalar) product is 
aTg — a 
éj6=6, (j=1,2,...,n) 


where 6;, is the Kronecker delta defined in Section 1.3.2. 
The set of normalized eigenvectors of a symmetric matrix therefore forms an ortho- 
normal set (that is, it forms a mutually orthogonal normalized set of vectors). 


Obtain the eigenvalues and corresponding orthogonal eigenvectors of the symmetric 
matrix 


2 2 0 
A=|2 5 0 
0 0 3 
and show that the normalized eigenvectors form an orthonormal set. 
The eigenvalues of A are A, = 6, A, = 3 and A, = 1, with corresponding eigenvectors 
a=) 2. 0]; @=[0 0 If, e,=[-2 1 oy 
which in normalized form are 
A= 2 ONS, A=(0 0 if, é,=[-2 1 Ooy'//5 


Evaluating the inner products, we see that, for example, 


aT a aT a 
éé,=1+i+0=1, 6,é,=-2+2+0=0 
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and that 


ATA _ 
éé, = 6, 


(i, j= 1, 2,3) 


confirming that the eigenvectors form an orthonormal set. 


1.4.8 Exercises 


Check your answers using MATLAB or MAPLE whenever possible. 


10 Verify Properties 


11 — Given that the eigenvalues of the matrix 


1.1—1.7 of Section 1.4.6. 12 Determine the eigenvalues and corresponding 
eigenvectors of the symmetric matrix 


-3 -3 -3 
4 A=/-3 1 -1l 
A=|2 5 4 =a.) 1 
=) =i 0 and verify that the eigenvectors are mutually 
orthogonal. 
are 5, 3 and 1: 
13. The 3 x 3 symmetric matrix A has eigenvalues 6, 
(a) confirm Properties 1.1—1.4 of Section 3 and 2. The eigenvectors corresponding to 
1.4.6; the eigenvalues 6 and 3 are[1 1 2]' and 
(b) taking k = 2, confirm Properties 1.5—1.7 of [1 1 —-1]' respectively. Find an eigenvector 


Section 1.4.6. 


1.5.1 


corresponding to the eigenvalue 2. 


Numerical methods 


In practice we may well be dealing with matrices whose elements are decimal numbers 
or with matrices of high orders. In order to determine the eigenvalues and eigenvectors 
of such matrices, it is necessary that we have numerical algorithms at our disposal. 


The power method 


Consider a matrix A having n distinct eigenvalues A,, A,,..., A,, and corresponding 
n linearly independent eigenvectors e,, @,,..., e,. Taking this set of vectors as the 
basis, we can write any vector x = [x, x, ... x,]' as a linear combination in the 
form 


n 
X= Me, + He t+...+4,e, = > ae; 
i=1 


Then, since Ae, = A,e, fori=1,2,...,n, 


n n 
Ax =A Y ae; = Y a;A;e; 
i=1 i=1 


www.2Ofile.org 


1.5 NUMERICAL METHODS 31 


and, for any positive integer k, 


n 
k k 
A x= » aA; e; 


i=l 


or 
k k ~ An’ 
A’x = A;| oe, + py a3) e; (1.12) 


Assuming that the eigenvalues are ordered such that 
|JA,| >A.) >... > 12,1 
and that a, # 0, we have from (1.12) 


lim A‘x = Ahare, (1.13) 
k-y0o 
since all the other terms inside the square brackets tend to zero. The eigenvalue A, and 
its corresponding eigenvector e, are referred to as the dominant eigenvalue and eigen- 
vector respectively. The other eigenvalues and eigenvectors are called subdominant. 
Thus if we introduce the iterative process 


xD = A (kK=0,1,2,...) 


(0) 


starting with some arbitrary vector x“? not orthogonal to e,, it follows from (1.13) 


that 
x = AKO 


will converge to the dominant eigenvector of A. 

A clear disadvantage with this scheme is that if |A,| is large then A‘x will become 
very large, and computer overflow can occur. This can be avoided by scaling the vector 
x” after each iteration. The standard approach is to make the largest element of x 
unity using the scaling factor max(x), which represents the element of x having the 
largest modulus. 

Thus in practice we adopt the iterative process 


(0) 


yD = Ax 


(k+1) 


Se 0) (1.14) 
(k+1) 
max(y  ’) 
and it is common to takhex®=[1 1 ... I]'. 


Corresponding to (1.12), we have 


n y k 
(k) R k i 
+ | <= : 
x Ay ae; 3 af ‘ e; 


where 


R= [max()?)max(y”) ... max(y)]! 
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Figure 1.1 Outline 
pseudocode program 
for power method to 
calculate the maximum 
eigenvalue. 


Example 1.10 


Solution 


Again we see that x converges to a multiple of the dominant eigenvector e,. Also, 
since Ax > 1.x, we have y“*) > 1,x™, and since the largest element of x” is unity, 
it follows that the scaling factors max(y“*”) converge to the dominant eigenvalue J,. 
The rate of convergence depends primarily on the ratios 


A,| [As A, 
Ay| | Ay A, 
The smaller these ratios, the faster the rate of convergence. The iterative process repres- 


ents the simplest form of the power method, and a pseudocode for the basic algorithm 
is given in Figure 1.1. 


> peg 


Hiteadinexel==| [Xap xquneexallt 
m<c0O0 
repeat 
mold — m 
{evaluate y = Ax} 
{find m = max(y;) } 
{x" = [y,/m y,/m... y,/m]} 
until abs(m — mold) < tolerance 
{write (results) } 


Use the power method to find the dominant eigenvalue and the corresponding eigen- 
vector of the matrix 


1 1 -2 
A=/-1 2 1 
0 1 -!l 


Taking xX =[1 1 1]' in (1.14), we have 


fod 2 /a). lo 0 

gS Ax’ = |=) 2 TN\t Si2H2i tls Ay? 
0 1 -1)/1] Jo 0 
0 

= 1yM = 1 
0 
1 1 -2//0] |1 0.5 

yO Sas Set 2 A | ile) S310 le a2 
Oo 1 -1}/0] |1 0.5 


BR 
i 
Nie 
<= 
s 
I | 
Ni — Nie | 
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Solution 


1 1 -2 
yosAx?=|-1 2° 1 
01 -l 
0.25 
eo S| 
0.25 
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ee 0.25 
1S S21 : a =2 
ae 0.25 


Continuing with the process, we have 


y® =2[0.375 1 0.375] 
y= 20.312 1 0.312] 
y = 2[0.344 1 0.344)" 
y” = 2[0.328 1 0.328] 
y® = 2[0.336 1 0.336] 


Clearly y® is approaching the vector 2[} 1 ay , so that the dominant eigenvalue is 


2 and the corresponding eigenvector is [ 


obtained in Example 1.4. 


Find the dominant eigenvalue of 


1 0 -1 0 
au|o 1 1 0 
-1 1 2 1 
00 1 -l 


; 1 ae which conforms to the answer 


Starting withx®=[1 1 1 1]', the iterations give the following: 


3 4 5 6 7 


3.3750 3.0741 3.2048 3.1636 3.1642 


Iteration k 1 2 
Eigenvalue —-— 3 2.6667 
x 1 0 -0.3750 
x? 1 0.6667 0.6250 
x 11 1 
x 1 0 0.3750 


4 


—0.4074 —-0.4578 —-0.4549 —0.4621 —0.4621 
0.4815 0.4819 0.4624 0.4621 0.4621 
1 1 1 1 1 
0.1852 0.2651 0.2293 0.2403 0.2401 


This indicates that the dominant eigenvalue is aproximately 3.16, with corresponding 
eigenvector [—0.46 0.46 1 0.24]'. 
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Example 1.12 


Solution 


The power method is suitable for obtaining the dominant eigenvalue and cor- 
responding eigenvector of a matrix A having real distinct eigenvalues. The smallest 
eigenvalue, provided it is non-zero, can be obtained by using the same method on the 
inverse matrix A when it exists. This follows since if Ax = Ax then A'x = 1'x. To 
find the subdominant eigenvalue using this method the dominant eigenvalue must first 
be removed from the matrix using deflation methods. We shall illustrate such a method 
for symmetric matrices only. 

Let A be a symmetric matrix having real eigenvalues 2,,2,,...,A,,. Then, by result 
(b) of Section 1.4.7, it has n corresponding mutually orthogonal normalized eigen- 
vectors é,, é,..., &, such that 


éjé=6, (i,j=1,2,...,7) 
Let A, be the dominant eigenvalue and consider the matrix 
A,=A-1,é,é) 
which is such that 
A,é, = (A- 1,6,é)é, = Aé, — 1,é(é1é,) = Né, — Xé, =0 
A,é, = Aé, — 1,é,(é76,) a 1,€; 


A é,= Aé, 7 1,€(e1&) = 1,6; 


aA A ALATA) _ A 
A en ~ Aé, = A,é\(é(é,) ~~ KE, 


Thus the matrix A, has the same eigenvalues and eigenvectors as the matrix A, except 
that the eigenvalue corresponding to A, is now zero. The power method can then be 
applied to the matrix A, to obtain the subdominant eigenvalue /, and its corresponding 
eigenvector e,. By repeated use of this technique, we can determine all the eigenvalues 
and corresponding eigenvectors of A. 


Given that the symmetric matrix 


2 
A=/|2 
0 


CoM WN 


0 
0 
3 
has a dominant eigenvalue A, = 6 with corresponding normalized eigenvector é, = 


[1 2  0]'/\5, find the subdominant eigenvalue 2, and corresponding eigenvector é). 


Following the above procedure, 


A,=A-1,éé1 
22 0 1 oe 
6 
=/2 5 o|-£/2|{1 2 oy=/-2 3} 0 
0 0 3 0 0 0 3 
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Applying the power method procedure (1.14), withx® =[1 1  1]', gives 


2 2 
5 15: 
y= Ax =|-1)=3/-b]; Ap? =3 
| 3 1 
2 0.133 
x =|—4] = | -0.133 
| 1 1 
| zy] s 
15 45 
yp? = Ax”? ae ee Ch 
| 3] 1 
2 0.044 
(2) _ 
x” =|-2] =| -0.044 
1 1 
[2] 2 
45 135 
y® Ax” 7 3 a > Ap = 8 
3 | 1 
0.015 
x” =| -0.015 
1 


Clearly the subdominant eigenvalue of A is 2, = 3, and a few more iterations confirm 
the corresponding normalized eigenvector as é,=[0 0 1)’. This is confirmed by the 
solution of Example 1.9. Note that the third eigenvalue may then be obtained using 
Property 1.1 of Section 1.4.6, since 


trace A= 10 =A, 4+1,4+1,;=64+3+4+A, 


giving 2, = 1. Alternatively, 7, and é, can be obtained by applying the power method 
to the matrix A, = A, — 16,65. 


Although it is good as an illustration of the principles underlying iterative methods 
for evaluating eigenvalues and eigenvectors, the power method is of little practical im- 
portance, except possibly when dealing with large sparse matrices. In order to evaluate 
all the eigenvalues and eigenvectors of a matrix, including those with repeated eigen- 
values, more sophisticated methods are required. Many of the numerical methods avail- 
able, such as the Jacobi and Householder methods, are only applicable to symmetric 
matrices, and involve reducing the matrix to a special form so that its eigenvalues can 
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1.5.2 


Theorem 1.1 


Theorem 1.2 


be readily calculated. Analogous methods for non-symmetric matrices are the LR and 
QR methods. It is methods such as these, together with others based on the inverse 
iterative method, that form the basis of the algorithms that exist in modern software 
packages such as MATLAB. Such methods will not be pursued further here, and the 
interested reader is referred to specialist texts on numerical analysis. 


Gerschgorin circles 


In many engineering applications it is not necessary to obtain accurate approximations 
to the eigenvalues of a matrix. All that is often required are bounds on the eigenvalues. 
For example, when considering the stability of continuous- or discrete-time systems 
(see Sections 5.7—6.8), we are concerned as to whether the eigenvalues lie in the 
negative half-plane or within the unit circle in the complex plane. (Note that the eigen- 
values of a non-symmetric matrix can be complex.) The Gerschgorin theorems often 
provide a quick method to answer such questions without the need for detailed calcula- 
tions. These theorems may be stated as follows. 


First Gerschgorin theorem 
Every eigenvalue of the matrix A = [a,], of order n, lies inside at least one of the 
circles (called Gerschgorin circles) in the complex plane with centre a,; and radii 
r= Lia joi |G] @= 1, 2, ..., 2). Expressed in another form, all the eigenvalues of the 
matrix A = [a,] lie in the union of the discs 
|z-a;| Sr; = by la;| (@=1,2,...,n) 

j=l 

j#i 
in the complex z plane. 


end of theorem 


Second Gerschgorin theorem 


If the union of s of the Gerschgorin circles forms a connected region isolated from the 
remaining circles then exactly s of the eigenvalues lie within this region. 


end of theorem 


Since the disc |z — a,;| <r; is contained within the disc 


n 
Iz| S|a,;|+7r;= > |a,;| 
j=l 


centred at the origin, we have a less precise but more easily applied criterion that all the 
eigenvalues of the matrix A lie within the disc 


|z| max| au (GHA, Doss. d 


j=l 


,N) (1.15) 


centred at the origin. 
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Example 1.13 


Solution 


Figure 1.2 
Gerschgorin circles 
for the matrix A of 
Example 1.13. 


The spectral radius (A) of a matrix A is the modulus of its dominant eigenvalue; 
that is, 


p(A) = max{|A,]} @=1,2,...,”) (1.16) 


where 1,,2,,...,4,, are the eigenvalues of A. Geometrically, p(A) is the radius of the 
smallest circle centred at the origin in the complex plane such that all the eigenvalues 
of A lie inside the circle. It follows from (1.15) that 


p(A) <= max| | (CS 1,223.6) (1.17) 


j=l 


Draw the Gerschgorin circles corresponding to the matrix 
10 -1 
A=|-1 2 
0 2 


What can be concluded about the eigenvalues of A? 


The three Gerschgorin circles are 
(i) |z-—10/=|-1|+0=1 
(ii) |z—2|=|-1]+|2|=3 
(iii) |z-—3|=|2|=2 
and are illustrated in Figure 1.2. 
It follows from Theorem 1.2 that one eigenvalue lies within the circle centred (10, 0) 
of radius 1, and two eigenvalues lie within the union of the other two circles; that is, 


within the circle centred at (2, 0) of radius 3. Since the matrix A is symmetric, it follows 
from result (a) of Section 1.4.7 that the eigenvalues are real. Hence 


9<A,< 11 


-1 < {A,,A3} <5 


23 4,5 6 7 8 910-11 12 


z plane 
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14 


1) 


16 


Wi, 


1.5.3 Exercises 


Use the power method to estimate the dominant 
eigenvalue and its corresponding eigenvector for 
the matrix 


4 3 
A=|3 5 
2 2 


ee NN YN 


Stop when you consider the eigenvalue estimate is 
correct to two decimal places. 


Repeat Exercise 14 for the matrices 


[2 1 0 3 0 1 
(a) A=|1 2 1 (b) A=|2 2 2 
I} 1 2 425 
[2 -1 0 
(©) A= -l1 2 -1 O 
0 -l1 2 -1 
|o © +1 2 


The symmetric matrix 


3 1 
A=/1 3 1 
1 5 


has dominant eigenvectore,;=[1 1 2]'. 
Obtain the matrix 


A,=A-166! 


where A, is the eigenvalue corresponding to the 
eigenvector e,. Using the deflation method, obtain 
the subdominant eigenvalue 1, and corresponding 
eigenvector e, correct to two decimal places, taking 
[1 1 1]'asa first approximation to e,. Continue 
the process to obtain the third eigenvalue 1, and its 
corresponding eigenvector e;. 


Draw the Gerschgorin circles corresponding to 
the matrix 


Sf =1 
A=| 1 0 1 
-1 1 -5 


18 


if) 


and hence show that the three eigenvalues are 
such that 


3<A4,<7, 2<A,<2, -7<1,<-3 


Show that the characteristic equation of the 
matrix 


10 -1 0O 
A=|-1 2 2 
0 2 3 


of Example 1.13 is 
f(A) =# - 154 + 51A-17=0 
Using the Newton—Raphson iterative procedure 


CD) 
Anu = A,- ; 
f'(An) 


determine the eigenvalue identified in 
Example 1.13 to lie in the interval 9< A< 11, 
correct to three decimal places. 

Using Properties 1.1 and 1.2 of Section 1.4.6, 
determine the other two eigenvalues of A to the 
same approximation. 


(a) If the eigenvalues of the n x n matrix A are 
A, >A, >A;,...4, 290 


show that the eigenvalue 1,, can be found by 
applying the power method to the matrix kI— A, 
where / is the identity matrix and k > A,. 


(b) By considering the Gerschgorin circles, show 
that the eigenvalues of the matrix 


2, i 0 
A=|-1 2 =1 
0 -l 2 


satisfy the inequality 
0<AS<4 


Hence, using the result proved in (a), determine 
the smallest modulus eigenvalue of A correct to 
two decimal places. 
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Reduction to canonical form 


In this section we examine the process of reduction of a matrix to canonical form. 
Specifically, we examine methods by which certain square matrices can be reduced or 
transformed into diagonal form. The process of transformation can be thought of as a 
change of system coordinates, with the new coordinate axes chosen in such a way that 
the system can be expressed in a simple form. The simplification may, for example, be 
a transformation to principal axes or a decoupling of system equations. 

We will see that not all matrices can be reduced to diagonal form. In some cases we 
can only achieve the so-called Jordan canonical form, but many of the advantages of 
the diagonal form can be extended to this case as well. 

The transformation to diagonal form is just one example of a similarity transform. 
Other such transforms exist, but, in common with the transformation to diagonal form, 
their purpose is usually that of simplifying the system model in some way. 


Reduction to diagonal form 


For an n X n matrix A possessing a full set of n linearly independent eigenvectors 
€), @,..., &, we can write down a modal matrix M having the n eigenvectors as its 
columns: 


M=[e, e e ... @] 


The diagonal matrix having the eigenvalues of A as its diagonal elements is called 
the spectral matrix corresponding to the modal matrix M of A, often denoted by A. 
That is, 


with the ijth element being given by 4,6,, where 6, is the Kronecker delta and i, j = 1, 
2,...,n. It is important in the work that follows that the pair of matrices Mand A 
are written down correctly. If the ith column of M is the eigenvector e, then the 
element in the (i, i) position in A must be /,, the eigenvalue corresponding to the 


eigenvector e,. 


We saw in Section 1.4.2 that in MATLAB the command 
[M,S]=eig (A) 


generates the modal and spectral matrices for the matrix A (Note: For convenience 
S is used to represent A when using MATLAB; whilst both are produced by the 
command Eigenvalues (A) in MAPLE.) 
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Example 1.14 


Solution 


Obtain a modal matrix and the corresponding spectral matrix for the matrix A of 
Example 1.4. 


1 1 -2 
A=|-1 2 1 
0 1 -i 


having eigenvalues 2, = 2, 2, = 1 and A, =—1, with corresponding eigenvectors 
e=(1 3 17, e=B 2 17, eg=[1 0 17 
Choosing as modal matrix M=[e, e, e,]' gives 
1 3 1 
M=)3 2 0 
1 1 1 


The corresponding spectral matrix is 


Returning to the general case, if we premultiply the matrix M by A, we obtain 


AM=Ale, e, ... e,)=[Ae, Ae, ... Ae] 
=[Aje, Ase ... A,e, 
so that 
AM= MA (1.18) 
Since the 1 eigenvectors e,, €,,..., e, are linearly independent, the matrix M is non- 


singular, so that M7 exists. Thus premultiplying by M™ gives 
M'AM= M'MA=A (1.19) 


indicating that the similarity transformation M~'AM reduces the matrix A to the diag- 
onal or canonical form A. Thus a matrix A possessing a full set of linearly independent 
eigenvectors is reducible to diagonal form, and the reduction process is often referred 
to as the diagonalization of the matrix A. Since 


A=MAM" (1.20) 
it follows that A is uniquely determined once the eigenvalues and corresponding eigen- 
vectors are known. Note that knowledge of the eigenvalues and eigenvectors alone is 


not sufficient: in order to structure M and A correctly, the association of eigenvalues 
and the corresponding eigenvectors must also be known. 
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Solution 
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Verify results (1.19) and (1.20) for the matrix A of Example 1.14. 


Since 
1 3 —2 2 2 
M=)3 2 O| wehave M' =~ 3 0 -3 
1 1 -l -2 7 
Taking 
2 0 O 
A=/0 1 0 
0 0 -!l 


matrix multiplication confirms the results 


M'AM= A, A=MAM" 


For an n X n symmetric matrix A it follows, from result (b) of Section 1.4.7, that 
to the n real eigenvalues A,, A5,..., A,, there correspond n linearly independent 
normalized eigenvectors é,, é),..., é, that are mutually orthogonal so that 


é:é=6, (,j=1,2,...,n) 
The corresponding modal matrix 
M=[é & ... é] 
is then such that 


| or [é; eo es é, | aT 


A al a aT 

e; ee; &) ey e1e), 
pile) me aT al a al a al a 
MéiM= ey _ €,€, @,@, ... e,e, 

al al a al a al a 

e, ene) €,e2 Cn en 

1 0 0 

0 1 0 

0 0... 1 


That is, M'M = land so M’ = M~. Thus Mis an orthogonal matrix (the term ortho- 
normal matrix would be more appropriate, but the nomenclature is long established). 

It follows from (1.19) that a symmetric matrix A can be reduced to diagonal form A 
using the orthogonal transformation 


MAM =A (1.21) 
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Example 1.16 


Solution 


1.6.2 


For the symmetric matrix A considered in Example 1.9 write down the corresponding 
orthogonal modal matrix M and show that M "AM = A, where A is the spectral matrix. 


From Example 1.9 the eigenvalues are A, = 6, 2, = 3 and A, = 1, with corresponding 
normalized eigenvectors 


é,=[1 2 o/5, &=[0 0 1, &=[-2 1 O75 


The corresponding modal matrix is 


V3 0 -2y; 
M=|2/1 0 i 
0 1 0 


and, by matrix multiplication, 


6 0 0 
MAM=|0 3 O|=A 
00 1 


The Jordan canonical form 


If an n X n matrix A does not possess a full set of linearly independent eigenvectors 
then it cannot be reduced to diagonal form using the similarity transformation M' AM. 
In such a case, however, it is possible to reduce A to a Jordan canonical form, making 
use of ‘generalized’ eigenvectors. 

As indicated in (1.11), if a matrix A has an eigenvalue /; of algebraic multiplicity 
m, and geometric multiplicity g,, with 1 < gq; < m,, then there are q; linearly independent 
eigenvectors corresponding to A ;. Consequently, we need to generate m; — g; generalized 
eigenvectors in order to produce a full set. To obtain these, we first obtain the q; linearly 
independent eigenvectors by solving 


(A-A,le,=0 


Then for each of these vectors we try to construct a generalized eigenvector e* such 
that 


(A-ANe*=e, 


If the resulting vector e* is linearly independent of all the eigenvectors (and generalized 
eigenvectors) already found then it is a valid additional generalized eigenvector. If 
further generalized eigenvectors corresponding to A; are needed, we then repeat the 
process using 


(A-Ade#* = ef 


and so on until sufficient vectors are found. 
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Solution 
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Obtain a generalized eigenvector corresponding to the eigenvalue A = 2 of Example 1.7. 
For 


1 
A= 0 
-1 


NY NY bw 
NO ee bv 


we found in Example 1.7 that corresponding to the eigenvalue 1, = 2 there was only 
one linearly independent eigenvector 


e=[2 1 oF 


and we need to find a generalized eigenvector to produce a full set. To obtain the general- 
ized eigenvector e¥, we solve 


(A - 2) )e¥=e, 
that is, we solve 
-1 2 2)) e& 2 
0 0 I}leSs=}1 
-1 2 O}} e& 0 


At once, we have e#, = | and e# = 2e%, and so 

era. 1 aj* 
Thus, by including generalized eigenvectors, we have a full set of eigenvectors for the 
matrix A given by 


e=[2 1 OJ* e=[2 1 1, e=f1 -1 17 


If we include the generalized eigenvectors, it is always possible to obtain for 
an n X n matrix A a modal matrix M with n linearly independent columns @é,, e,, 
..., &,. Corresponding to (1.18), we have 

AM= MJ 


where J is called the Jordan form of A. Premultiplying by M™ then gives 
M'AM= J (1.22) 


The process of reducing A to J is known as the reduction of A to its Jordan normal, or 
canonical, form. 
If A has p distinct eigenvalues then the matrix J is of the block-diagonal form 


J=[J, J, ... Ji 
where each submatrix J, (i= 1, 2,..., p) is associated with the corresponding eigen- 
value A,. The submatrix J, will have A, as its leading diagonal elements, with zeros 


elsewhere except on the diagonal above the leading diagonal. On this diagonal the 
entries will have the value | or 0, depending on the number of generalized eigenvectors 
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Example 1.18 


Solution 


used and how they were generated. To illustrate this, suppose that A is a 7 x 7 matrix 
with eigenvalues A, = 1, A, = 2 (occurring twice), 2, = 3 (occurring four times), and 
suppose that the number of linearly independent eigenvectors generated in each case is 


A,=1, 1 eigenvector 
A,=2, 1 eigenvector 
A,;=3, 2 eigenvectors 


with one further generalized eigenvector having been determined for 2, = 2 and two 
more for A; = 3. 

Corresponding to A, = 1, the Jordan block J, will be just [1], while that corresponding 
to A, =2 will be 


Corresponding to A, = 3, the Jordan block J; can take one of the two forms 


aA, 1 0 : 0 fa, 1 + 0 0 
0 A, 1! 0 0 Feo @ 
J3;=|0 O A, ! 0 or Jz. = |---- ccc prt 
vane eens eee Oo 2 a ft 
0 0 0 A5| 0 0 A; 


depending on how the generalized eigenvectors are generated. Corresponding to A, = 3, 
we had two linearly independent eigenvectors e;, and e; 5. If both generalized eigen- 
vectors are generated from one of these vectors then J, will take the form J; ,, whereas 
if one generalized eigenvector has been generated from each eigenvector then J, will 
take the form J;). 


Obtain the Jordan canonical form of the matrix A of Example 1.17, and show that 
MAM = J where Mis a modal matrix that includes generalized eigenvectors. 


For 
1 2 2 
A=/; 0 2 1 
-1 2 2 


from Example 1.17 we know that the eigenvalues of A are A, = 2 (twice) and A, = 1. 
The eigenvector corresponding to 7, = 1 has been determined as e;=[1 1 —1]' in 
Example 1.7 and corresponding to A, = 2 we found one linearly independent eigen- 
vectore,=[2 1 O]' anda generalized eigenvector e*=[2 1 1]'. Thus the modal 
matrix including this generalized eigenvector is 


2 2 1 
M=j)1 1 1 
0 1 -!l 
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and the corresponding Jordan canonical form is 


To check this result, we compute M™ as 


a | 
M'=|-1 2 1 
-1 2 0 


and, forming MAM, we obtain J as expected. 


In MATLAB the command J=jordan(A) computes the Jordan form of A; including 
the case when J is diagonal and all the eigenvectors of A are linearly independent. 
The command 


[M,J]=jordan (A) 


also computes the similarity transformation or modal matrix M that may include 
generalized eigenvectors. 

Numerical calculation of the Jordan form is very sensitive to round-off errors, etc. 
This makes it very difficult to compute the Jordan form reliably and almost any 
change in A causes it to be diagonal. 

For the matrix A in Example 1.18 the sequence of commands 

Dap 2 2,0 2 tp =i 2 Di- 
[M,J]=jordan (A) 


returns 
=i =2 2 
Mest =a) iL 
iL @ sil 
iL 0) 0 
Je 0 2 dl 
0 0 2 


which is equally acceptable to the solution given in Example 1.18. (This can be 
checked by evaluating MAM.) 
Using the Symbolic Math Toolbox in MATLAB the sequence of commands 
Mei 2 Be @ 2 ig =i 2 Zlle 
AS=sym A 
(M, J] =jordan (AS) 
returns the same output as above. In practice, this sequence of commands is only 
really effective when the elements of the matrix A are integers or ratios of small 
integers. 
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20 


Zl 


22 


23 


1.6.3 Exercises 


Check your answers using MATLAB or MAPLE whenever possible. 


Show that the eigenvalues of the matrix 24 
=1 6 -12 
A=| 0 -13 30 
0 -9 20 


are 5, 2 and —1. Obtain the corresponding 
eigenvectors. Write down the modal matrix 
M and spectral matrix A. Evaluate M™ and 
show that M"'AM= A. 


Using the eigenvalues and corresponding 25 
eigenvectors of the symmetric matrix 
2 2. 0 
A=|2 5 0 
0 0 3 
26 


obtained in Example 1.9, verify that 
MM" AM = A where Mand A are respectively 
a normalized modal matrix and a spectral 


matrix of A. 
Given 
5 10 8 
A=/10 2 -2 
8 —2 11 


find its eigenvalues and corresponding 

eigenvectors. Normalize the eigenvectors 

and write down the corresponding normalized 27 
modal matrix M. Write down M" and show 

that M"AM = A, where A is the spectral 

matrix of A. 


Determine the eigenvalues and corresponding 
eigenvectors of the matrix 


1 —2 
A=|-1 2 1 
0 1 -il 


Write down the modal matrix M and spectral 
matrix A. Confirm that M“'AM = A and that 
A= MAM". 


Determine the eigenvalues and corresponding 
eigenvectors of the symmetric matrix 


3 2 4 
A=|-2 -2 6 
4 6 -l 


Verify that the eigenvectors are orthogonal, 
and write down an orthogonal matrix L such that 
LTAL = A, where A is the spectral matrix of A. 


A 3x3 symmetric matrix A has eigenvalues 
6, 3 and 1. The eigenvectors corresponding 

to the eigenvalues 6 and 1 are[1 2 O]' and 
[-2 1 OJ’ respectively. Find the eigenvector 
corresponding to the eigenvalue 3, and hence 
determine the matrix A. 


Given that A = | is a thrice-repeated eigenvalue 
of the matrix 


3-7 5 
A=|2 4 3 
i 2 2 


use the nullity, given by (1.11), of a suitable matrix 
to show that there is only one corresponding linearly 
independent eigenvector. Obtain two further 
generalized eigenvectors, and write down the 
corresponding modal matrix M. Confirm that MAM 
= J, where J is the appropriate Jordan matrix. 


Show that the eigenvalues of the matrix 


1 0 0-3 

a-|® 1-3 0 
-05 -3 1 05 
3 0 0 1 


are —2, —2, 4 and 4. Using the nullity, given 

by (1.11), of appropriate matrices, show that 
there are two linearly independent eigenvectors 
corresponding to the repeated eigenvalue —2 
and only one corresponding to the repeated 
eigenvalue 4. Obtain a further generalized 
eigenvector corresponding to the eigenvalue 4. 
Write down the Jordan canonical form of A. 
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Example 1.19 


Solution 
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Quadratic forms 


A quadratic form in 1 independent variables x,, x), ... , x, 1s a homogeneous second- 
degree polynomial of the form 


n n 
VX jy X54 55%) = S Y AX; X; 
i=l j=l 
_ 2 
= Ay Xt AyXyX, +... +A), XX, 


2 
HF Ag XX F Ag X94 +... F Any XX, 


2 
FA hy XH QypXgXo F vs F Upp X; (1.23) 
Defining the vectorx=[x, x, ... x,]' and the matrix 
ay ay? ea-% Qin 
Aa|% G2 ++ Ge 
ant an2 aries Ann 


the quadratic form (1.23) may be written in the form 

V(x) =x'Ax (1.24) 
The matrix A is referred to as the matrix of the quadratic form and the determinant of 
A is called the discriminant of the quadratic form. 

Now a, and a; in (1.23) are both coefficients of the term x;x, (i #/), so that for i #7 
the coefficient of the term x,x; is a, + a. By defining new coefficients a; and a; for x;x, 
and x,x; respectively, such that aj = aj,= 5 (a; + a,;), the matrix A associated with the 
quadratic form V(x) may be taken to be symmetric. Thus for real quadratic forms we 
can, without loss of generality, consider the matrix A to be a symmetric matrix. 


Find the real symmetric matrix corresponding to the quadratic form 


V(x1, Xp, X3) =X} + 3x5 — Ax} — 3x,x, + 2x11, — Sxyx; 


Ifx=[x, x, %3]’, we have 


3 2 
Po-3 5] pm 
~ 3 5 ask 
V(x), X2,%3) =[x, xX. X35] me) 3. -5)| x2.) =x Ax 
ee eee 
5 3 4 X3 


i= s 
A=|3 3 -3 
1-2 -4 
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Example 1.20 


Solution 


In Section 1.6.1 we saw that a real symmetric matrix A can always be reduced to the 
diagonal form 


M'AM=A 


where M is the normalized orthogonal modal matrix of A and A is its spectral matrix. 
Thus for a real quadratic form we can specify a change of variables 


x= My 
wherey=[y, y. ... y,]', Such that 
V=x'Ax = y'M' AMy = y'Ay 
giving 
V = Ait Ayot...tAwe (1.25) 


Hence the quadratic form x"Ax may be reduced to the sum of squares by the trans- 
formation x = M 'y, where M is the normalized modal matrix of A. The resulting form 
given in (1.25) is called the canonical form of the quadratic form V given in (1.24). 
The reduction of a quadratic form to its canonical form has many applications in 
engineering, particularly in stress analysis. 


Find the canonical form of the quadratic form 
V = 2xj + 5x5 + 3x3 + 4x, 


Can V take negative values for any values of x,, x, and x;? 


At once, we have 


2 2 0 
V=x'|2 5 Olx=x'Ax 
0 0 3 


where 


2 
x=[x, % ~u]', A=)2 
0 


oN NM 
wn Oo oO 


The real symmetric matrix A is the matrix of Example 1.16, where we found the 
normalized orthogonal modal matrix M and spectral matrix A to be 


6 0 0 
, A=|0 3 0 
0 0 1 
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such that M'AM = A. Thus, setting x = My, we obtain 


6 0 0 
V=y'M'AMy=y'|0 3 Oly = 6y 4393495 
001 


as the required canonical form. 

Clearly V is non-negative for all y,, y, and y;. Since x = M y and Mis an orthogonal 
matrix it follows that y = Mx, so for all x there is a corresponding y. It follows that V 
cannot take negative values for any values of x,, x, and x;. 


The quadratic form of Example 1.20 was seen to be non-negative for any vector x, 
and is positive provided that x # 0. Such a quadratic form x'Ax is called a positive- 
definite quadratic form, and, by reducing to canonical form, we have seen that this 
property depends only on the eigenvalues of the real symmetric matrix A. This leads us 
to classify quadratic forms V = x'Ax, where x = [x, x, ... x,]’ in the following 
manner. 


(a) Vis positive-definite, that is V > 0 for all vectors x except x = 0, if and only 
if all the eigenvalues of A are positive. 


(b) Vis positive-semidefinite, that is V => 0 for all vectors x and V = 0 for at least 
one vector x # 0, if and only if all the eigenvalues of A are non-negative and 
at least one of the eigenvalues is zero. 

(c) Vis negative-definite if —V is positive-definite, with a corresponding condition 
on the eigenvalues of —A. 

(d) Vis negative-semidefinite if—V is positive-semidefinite, with a corresponding 
condition on the eigenvalues of —A. 


(ce) V is indefinite, that is V takes at least one positive value and at least one 
negative value, if and only if the matrix A has both positive and negative 
eigenvalues. 


Since the classification of a real quadratic form x'Ax depends entirely on the location 
of the eigenvalues of the symmetric matrix A, it may be viewed as a property of A itself. 
For this reason, it is common to talk of positive-definite, positive-semidefinite, and so 
on, symmetric matrices without reference to the underlying quadratic form. 


Classify the following quadratic forms: 

(a) 3x7 + 2x5 + 3x3 — 2x,x, — 2x x; 

(b) 7x7 +x5 4x3 - 4x,x, — 4x,x, + 8x5x, 

(c) —3xj— 5x3 — 3x3 + 2x,x, + 2x, — 2x,x; 


(d) 4x7 4+x34 15x} - 4x,x, 
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Solution 


(a) The matrix corresponding to the quadratic form is 


3 -l 0 
A=|-1 2 -1 
0 -l 3 


The eigenvalues of A are 4, 3 and 1, so the quadratic form is positive-definite. 


(b) The matrix corresponding to the quadratic form is 


7 23 
A=|-2 1 4 
2 4 i 


The eigenvalues of A are 9, 3 and —3, so the quadratic form is indefinite. 


(c) The matrix corresponding to the quadratic form is 


-3 1 -l 
A=|1 -5 1 
-1 1 —-3 


The eigenvalues of A are —6, —3 and —2, so the quadratic form is negative-definite. 


(d) The matrix corresponding to the quadratic form is 


4 —2 0 
A=|-2 1 0 
0 O 15 


The eigenvalues of A are 15, 5 and 0, so the quadratic form is positive- 
semidefinite. 


In Example 1.21 classifying the quadratic forms involved determining the eigen- 
values of A. If A contains one or more parameters then the task becomes difficult, if not 
impossible, even with the use of a symbolic algebra computer package. Frequently in 
engineering, particularly in stability analysis, it is necessary to determine the range of 
values of a parameter k, say, for which a quadratic form remains definite or at least 
semidefinite in sign. J. J. Sylvester determined criteria for the classification of quadratic 
forms (or the associated real symmetric matrix) that do not require the computation of 
the eigenvalues. These criteria are known as Sylvester’s conditions, which we shall 
briefly discuss without proof. 

In order to classify the quadratic form x" Ax Sylvester’s conditions involve considera- 
tion of the principal minors of A. A principal minor P. of order i (i= 1, 2,..., 1) of 
an n Xn square matrix A is the determinant of the submatrix, of order 7, whose principal 
diagonal is part of the principal diagonal of A. Note that when i = 7 the principal minor 
is det A. In particular, the leading principal minors of A are 


Qi, Ajy2 43 
s D; =a, a2 Q93/ 5 «205 D, = det A 


43; A327 33 


Qi, Ay2 


D,=|a,|, D.= 


Az, Ang 
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Example 1.22 Determine all the principal minors of the matrix 


1 k O 
A=|k 2 0 
0 0 5 
and indicate which are the leading principal minors. 


Solution (a) The principal minor of order three is 
P,;= det A= 5(2—k*) (leading principal minor D,) 
(b) The principal minors of order two are 


(i) deleting row | and column 1, 


> 0 
Py = we = 10 
(ii) deleting row 2 and column 2, 
Py = oN =5 
0 5 
(iii) deleting row 3 and column 3, 
Py, = ‘ ‘ =2-k (leading principal minor D,) 


(c) The principal minors of order one are 


(i) deleting rows 1| and 2 and columns | and 2, 


P, =(|5|=5 
(ii) deleting rows | and 3 and columns | and 3, 
P= (2|=2 


(iii) deleting rows 2 and 3 and columns 2 and 3, 


P,,=|1]=1 (leading principal minor D,) 


Sylvester’s conditions: These state that the quadratic form x’Ax, where A is an 
n Xn real symmetric matrix, is 


(a) positive-definite if and only if all the leading principal minors of A are 
Positives thatise)ae-10\(i —sleeDeerneeerD): 

(b) negative-definite if and only if the leading principal minors of A alternate in 
sign with a,, < 0; that is, (-1)'D, > 0 (i@=1, 2,...,n); 

(c) positive-semidefinite if and only if det A = 0 and all the principal minors of 
A are non-negative; that is, det A = 0 and P, = 0 for all principal minors; 


(d) negative-semidefinite if and only if det A = 0 and (-1)'P, = 0 for all principal 
minors. 
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Example 1.23 


Solution 


Example 1.24 


Solution 


For what values of & is the matrix A of Example 1.22 positive-definite? 


The leading principal minors of A are 


D,=1, D,=2-k*, D,=5(2-k*) 


These will be positive provided that 2 — k* > 0, so the matrix will be positive-definite 
provided that k* < 2, that is —/2 <k < \2. 


Using Sylvester’s conditions, confirm the conclusions of Example 1.21. 


(a) 


(b) 


(c) 


The matrix of the quadratic form is 


3 -l1 O 
A=|-l1 2 -1l 
0 -l 3 


and its leading principal minors are 


3 i” 


3, | =5, detA=12 
-1 2 


Thus, by Sylvester’s condition (a), the quadratic form is positive-definite. 


The matrix of the quadratic form is 


a 
A=|-2 1 4 
3 gf i 


and its leading principal minors are 


7 -2 
qs =3, detA=-81 
—2 1 


Thus none of Sylvester’s conditions can be satisfied, and the quadratic form is 
indefinite. 


The matrix of the quadratic form is 


-3 1 -l 
A=); 1 -5 1 
-1 1 -3 


and its leading principal minors are 
3 1 | 


=14, det A=-36 
if 5 


-3, | 


Thus, by Sylvester’s condition (b), the quadratic form is negative-definite. 
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(d) The matrix of the quadratic form is 


4 2 0 
A=|-2 1 0 
0 oO 15 


and its leading principal minors are 


4. 2 
4, 
3 | 


=o, det A=0 


We therefore need to evaluate all the principal minors to see if the quadratic form 
is positive-semidefinite. The principal minors are 


de 22 


4, 1, 15, 
=z 1 


9 | 


0 
15 


4 0 


=15, 
| ; 15 


= 60, det A=0 


Thus, by Sylvester’s condition (c), the quadratic form is positive-semidefinite. 


1.6.5 Exercises 


Reduce the quadratic form 
2x7 + 5x5 + 2x3 + 4xxy + 2x5x, + 4KX> 


to the sum of squares by an orthogonal 
transformation. 


Classify the quadratic forms 
(a) x} + 2x3 + 7x3 — 2x,x) + 4x,x, — 2xyx, 


(b) x7 + 2x3 + 5x} — 2x,x) + 4xyx; — 2x x; 


(c) x7 + 2x3 + 4x3 — 2x,x, + 4x,x, — 2x,x, 


(a) Show that axj— 2bx,x, + cx} is positive-definite 
if and only if a > 0 and ac > Db’. 

(b) Find inequalities that must be satisfied by a and 
b to ensure that 2x} + ax3 + 3x} — 2x,x, + 2bx,x; 
is positive-definite. 


Evaluate the definiteness of the matrix 


2 dL -—l 
A=| 1 2 1 
-1l il 2 
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(a) by obtaining the eigenvalues; 
(b) by evaluating the principal minors. 


Determine the exact range of k for which the 
quadratic form 

O(x, y, Z) = k(x? + y?) + 2xy +27 + 2xz — 2yz 
is positive-definite in x, y and z. What can be said 


about the definiteness of O when k = 2? 


Determine the minimum value of the constant 
a such that the quadratic form 


34a 1 1 
x'| 1 a 2\x 
1 2 4a 


where x =[x, x, %,]', is positive-definite. 


Express the quadratic form 


O=x}+4x,x, 6xx; + A(xd +.x3) 


4x x3 


in the form x"Ax, wherex=[x, x, x,]' and 
A is a symmetric matrix. Hence determine 

the range of values of A for which Q is 
positive-definite. 
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Functions of a matrix 


Let A be ann X n constant square matrix, so that 
A’ = AA, A’ = AA’ = AA, and so on 


are all defined. We can then define a function /(A) of the matrix A using a power series 
representation. For example, 


f(A) = SBA’ = Bol+ B,A+...+ B,A’ (1.26) 


where we have interpreted A’ as the n x n identity matrix I. 


Example 1.25 — Given the 2 x 2 square matrix 
1 -1 
A — 


determine f(A) = > BA when B, = 1, B, =—1 and f, = 3. 


r=0 
Solution Now 


FB) = Bs Bs B= | *|-1)) wer 4 
0 


a|-s =) 
22 19 


Note that A is a 2 x 2 matrix and f(A) is another 2 x 2 matrix. 


Suppose that in (1.26) we let p > ©, so that 


fA) = > BB 
r=0 
We can attach meaning to f(A) in this case if the matrices 
Pp 
f(A) = ¥ BA 
r=0 


tend to a constant n x n matrix in the limit as p > . 
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Example 1.26 For the matrix 


using a computer and larger and larger values of p, we infer that 


Pp 5 
2 ee 

A) =1im> 4 = 

ner F Deel 


indicating that 


aA)=|° ° 
raf 4 


What would be the corresponding results if 
-1 0 -t 0 
A= ; b) A= ? 
(a) 0 j (b) ; j 


Solution (a) The computer will lead to the prediction 


f(A) = 


(71828) 0 
0 2718 28 


indicating that 


-1 
e 0 
f(A) = 
0 e 
(b) The computer is of little help in this case. However, hand calculation shows that 
we are generating the matrix 


1-r+1P-lP +... 0 
J(A) = 


0 l+rtiP tire... 


indicating that 
e' 0 
0 e 


By analogy with the definition of the scalar exponential function 


f(A) = 


22 mr r 
eVateare2ey. 424 ay 
2! r 
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Theorem 1.3 


Example 1.27 


Solution 


it is natural to define the matrix function e”’, where f is a scalar parameter, by the power 
series 


f(A) = ya (1.27) 


In fact the matrix in part (b) of Example 1.26 illustrates that this definition is reasonable. 

In Example 1.26 we were able to spot the construction of the matrix f(A), but this 
will not be the case when A is a general n x n square matrix. In order to overcome this 
limitation and generate a method that will not rely on our ability to ‘spot’ a closed form 
of the limiting matrix, we make use of the Cayley—Hamilton theorem, which may be 
stated as follows. 


Cayley—Hamilton theorem 


A square matrix A satisfies its own characteristic equation; that is, if 
RB +6, ,R 1 +6, +...+¢A+c¢=0 
is the characteristic equation of an n x n matrix A then 
A’ +c, A"! +¢,,4'?+...+¢A+cl=0 (1.28) 
where I is the n x n identity matrix. 
end of theorem 
The proof of this theorem is not trivial, and is not included here. We shall illustrate the 


theorem using a simple example. 


Verify the Cayley—Hamilton theorem for the matrix 
A-|> 4 
1 2 
The characteristic equation of A is 
| 3-A 4 


=0 or #-504+2=0 
1 2-A 


we have 


Aospeoi=|t? 28) 25)? Foo) Men 
5 8 1) 2 Oo 4 


thus verifying the validity of the Cayley—Hamilton theorem for this matrix. 
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In the particular case when A is a 2 x 2 matrix with characteristic equation 
cA) =H +aA+a,=0 (1.29) 
it follows from the Cayley—Hamilton theorem that 
(A) = A’+a4,A+a1=0 


The significance of this result for our present purposes begins to appear when we 
rearrange to give 


AB’ =-a, A- al 


This means that A’ can be written in terms of A and A’ = I. Moreover, multiplying by 
A gives 


A’ = -a,A’ — a,A=—a,(-a,A- a1) —-a,A 


Thus A’ can also be expressed in terms of A and A° = I; that is, in terms of powers of 
A less than n = 2, the order of the matrix A in this case. It is clear that we could continue 
the process of multiplying by A and substituting A’ for as long as we could manage the 
algebra. However, we can quickly convince ourselves that for any integer r = n 


A=a,1+a,A (1.30) 


where @, and @, are constants whose values will depend on r. 

This is a key result deduced from the Cayley—Hamilton theorem, and the determina- 
tion of the a; (i= 0, 1) is not as difficult as it might appear. To see how to perform the 
calculations, we use the characteristic equation of A itself. If we assume that the eigen- 
values A, and A, of A are distinct then it follows from (1.29) that 


CA) =A +a A;+a=0 (i=1, 2) 
Thus we can write 
X= -a,A,- a 
in which a, and a, are the same constants as in (1.29). Then, for 7 = 1, 2, 


B= -a,A?- ad, =-a,(-a,A,- a) - aA, 


Proceeding in this way, we deduce that for each of the eigenvalues 2, and 2, we 
can write 


N= Ay) + A; 


with the same O, and @, as in (1.30). This therefore provides us with a procedure for 
the calculation of AY when r = n (the order of the matrix) is an integer. 


Given that the matrix 


a=|° 1 
—2 -3 
has eigenvalues A, =—1 and 2, =—2 calculate A’ and A’, where r is an integer greater 


than 2. 
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Solution 


Example 1.29 


Solution 


Since A is a 2 x 2 square matrix, it follows from (1.30) that 
A=a,l+a,A 
and for each eigenvalue A, (i = 1, 2) a, and a, satisfy 
R= A)+ aA; 
Substituting 2, =—1 and 7, =—2 leads to the following pair of simultaneous equations: 
(-1)° = a) + a@,(-1), (-2)° = ay + a,(-2) 
which can be solved for @ and @, to give 
Oy =2-1P-(-2Y, a =(-1)- (2)? 
Then 


A’ acy C2)! *|+i'-ca| ° | 
0 1 —2 -3 


_ | 2-1)" - (27° (-1)°-(-2)° |_| 30 31 
(20-1 -@2)"), 20-2)*=(1)*| [-62 -63 
Replacing the exponent 5 by the general value r, the algebra is identical, and it is easy 
to see that 


A’ = 


2(-1)"- (-2y’ (=1) =(-2) | 
=A ly (=2)) 2627 1) 


To evaluate @, and a, in (1.27), we assumed that the matrix A had distinct eigen- 
values A, and A,, leading to a pair of simultaneous equations for @ and a. What 
happens if the 2 x 2 matrix A has a repeated eigenvalue so that A, = 1, = A, say? 
We shall apparently have just a single equation to determine the two constants @, and 
a,. However, we can obtain a second equation by differentiating with respect to 
A, as illustrated in Example 1.29. 


Given that the matrix 


a9 


has eigenvalues A, = 2, =—1, determine A’, where r is an integer greater than 2. 


Since A is a 2 x 2 matrix, it follows from (1.30) that 


A’=a,1+a,A 
with @, and @, satisfying 
N= A)+ A (1.31) 
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Since in this case we have only one value of A, namely 1 = —1, we differentiate (1.31) 
with respect to A, to obtain 


rx =a, (1.32) 
Substituting A =—1 in (1.31) and (1.32) leads to 
a=, y= (“1+ = (1 -r)C-lY 


giving 


a =c-neny| t|-rcn|? , 
0 1 = 


Jiri andy 
fly Ale Dy: 


Having found a straightforward way of expressing any positive integer power of the 
2 x 2 square matrix A we see that the same process could be used for each of the terms 
in (1.26) for r = 2. Thus, for a 2 x 2 matrix A and some @, and a, 


f(A) = y B.A = aI+a,A 


r=0 
If, as p > 9, 


F(A) = lim SBA 


exists, that is, it is a 2 x 2 matrix with finite entries independent of p, then we may write 
f(A) = » B.A =al+a,A (1.33) 


We are now in a position to check the results of our computer experiment with the matrix 


A= i of Example 1.26. We have defined 
0 1 


pany do $ By 


so we can write 
e’=~al+a,A 
Since A has repeated eigenvalue A = 1, we adopt the method of Example 1.29 to give 


e&b=A4,+ OH, te'=a, 
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Example 1.30 


Solution 


leading to 


a,=te’, a,=UA-de' 


t 
ef =(1-Nel +teA=el =| ° s 
0 e 


Setting ¢= 1 confirms our inference in Example 1.26. 


Calculate e*’ and sin At when 
eae 
0 1 


Again A has repeated eigenvalues, with 7, = A, = 1. Thus for e*’ we have 
c= a,l+a,A 

with 
b= A),+ %, te’ =a, 


leading to 


Similarly, 
sin At= al+ aA 
with 
sint = )+ O%, tcost= GQ, 


leading to 


sin At = sint -—tcost 
0 sin ¢ 


Although we have worked so far with 2 x 2 matrices, nothing in our development 
restricts us to this case. The Cayley—Hamilton theorem allows us to express positive 
integer powers of any n X n square matrix A in terms of powers of A up to n — 1. That 
is, if Ais ann Xn matrix and p = n then 


n-1 
A’= BA’ = Bol + BAt...+ B.A" 
r=0 
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From this we can deduce that for an n x n matrix A we may write 


fA) = > BN 
r=0 
as 
n-1 i 
SA)=> oA (1.34a) 
r=0 
which generalizes the result (1.33). Again the coefficients >, O),..., QO, are 


obtained by solving the n equations 


f(A) = ¥ aA! (=1,2,.2:,n) (1.34b) 


r=0 


where A,, 2,,..., A, are the eigenvalues of A. If A has repeated eigenvalues, we 
differentiate as before, noting that if A; is an eigenvalue of multiplicity m then the 
first m — 1 derivatives 


d‘ d‘ ioe 
are? aan Sia,d; (k= 1,2,...,m—1) 


D0) 


are also satisfied by A. 


Sometimes it is advantageous to use an alternative approach to evaluate 
Pp 
f(A) => BA 


r=0 


If A possesses n linearly independent eigenvectors then there exists a modal matrix M 
and spectral matrix A such that 


M"'AM= A= diag (A,, 72,...,4,) 
Now 


M'/(A)M= > B,(M'A'M) = > B,(M'AM)’ 


0 r=0 


Pp Pp 
= >» A = Y B, diag(Aj, As, ... , Aj) 
= 


P P P 
= diag b BAw > BA sets px 
r=0 r=0 


r=0 


a 


= diag (f(A,), f(A2), --- »f(A,)) 


This gives us a second method of computing functions of a square matrix, since we see that 


JA) = Mdiag ( f(A,), f(A2), ---.f(A,))M™ (1.35) 
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Example 1.31 — Using the result (1.35), calculate A* for the matrix 


[3 2 


of Example 1.28. 


Solution A has eigenvalues 2, = —1 and A, = —2 with corresponding eigenvectors 
e=[l -1]", e=[1 -2]" 


Thus a modal matrix Mand corresponding spectral matrix A are 


M= 2 1 
-l1 -1 


Taking f(A) = A‘, we have 
diag ( f(-1), f(-2)) = diag (-1), (-2)') 
Thus, from (1.35), 
(ly) 0 
0 (-2) 


2(-1)-(-2)"  (-1)*- (-2)" 
2((-2)"-(-1)‘) 2(-2)‘- (-1)" 


f(A)=M M ‘= 


k 


as determined in Example 1.28. 


Example 1.31 demonstrates a second approach to the calculation of a function of a 
matrix. There is little difference in the labour associated with each method, so perhaps 
the only comment we should make is that each approach gives a different perspective 
on the construction of the matrix function either from powers of the matrix itself or 
from its spectral and modal matrices. 

Later in this chapter we need to make use of some properties of the exponential 
matrix e*’, where A is a constant n x n square matrix. These are now briefly discussed. 
(i) | Considering the power series definition given in (1.27) 

eM=I+Ar+ LAr + Art... 


term-by-term differentiation gives 


Dy 2A APY... = AIG APE LAPS. 
dt 2! 3! 2! 
so that 
£ (Coe eas (1.36) 
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(ii) Likewise, term-by-term integration of the power series gives 


| evar] aval tdt + A'| Tdt+... 
0 0 0 ~ 0 


=It+ + Ar + sAt +. ~ 
so that 


a| e"drt+ I= e* 


0 


giving 


| edt = Ale — I] =[e™ — At (1.37) 


0 


provided the inverse exists. 


A(t, +t At, At, 
e (ty Dee 1Q%2 


(iii) (1.38) 


Although this property is true in general we shall illustrate its validity for the 
particular case when A has z linearly independent eigenvectors. Then, from (1.35), 


eM = Mdiag(e*"", et, ent)M | 
e“? = Mdiag(e*!?, 22... , e4")M! 
so that 
eight = Mdiag (71, eth) ue ef nt) Mp! — eAit) 
(iv) It is important to note that in general 


eft eB! # e(ArB yt 


It follows from the power series definition that 
eAt eBr = e(AtB)t (1.39) 
if and only if the matrices A and B commute; that is, if AB = BA. 
To conclude this section we consider the derivative and integral of a matrix A(t) = 


[a,(2)], whose elements a;,(t) are functions of ¢. The derivative and integral of A(¢) are 
defined respectively by 


d _|a 
fay = le al (1.402) 


and 


fav da | a(t) 7 (1.40b) 


that is, each element of the matrix is differentiated or integrated as appropriate. 
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Example 1.32 — Evaluate dA/d¢ and fAd¢ for the matrix 
f+1 t-3 
2 f£+2t-1 


Solution Using (1.40a), 


d,2 d 
—(t 1 ie = 
dA _ di! oe ai °) |? 1 
dt d d 2 0 2r+2 
= (f+2t-1 
a ae +2t-1) 


Using (1.40b), 


2 
[ac Jo +1)dt Jo 3) dt WP +tten 1? -3ttey 
t= = : 
3 2 
[a fieer Dae | el greene 
EF ; ‘a 123 2: 
-f +t =t' -3t Cy C12 ge +0 jt - 3t 
—/}3 2 + =,|4 2 +C 
2t Lf +f-1 Cy Cap 2t e +f -t 


where C is a constant matrix. 


(2) Using the Symbolic Math Toolbox in MATLAB the derivative and integral of the 
matrix A(t) is generated using the commands diff (A) and int (A) respectively. 
To illustrate this confirm that the derivative of the matrix A(¢) of Example 1.32 is 
generated using the sequence of commands 
syms t 
Me eres Vesa p 2 ioe <All i) 
df=diff (A); 
pretty (df) 

and its integral by the additional commands 
T=int (A); 
pretty (1) 


From the basic definitions, it follows that for constants @ and B 


d _ fA, p(B 

(cat BB)=a 7 +B a (1.41) 
| coms para aarp [Ba (1.42) 
d (apy)-adB, dA 

(AB) =A a Ve (1.43) 


Note in (1.43) that order is important, since in general AB # BA. 
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Note that in general 


d n n-1dA 
—lA znkh — 


1.7.1 Exercises 
Check your answers using MATLAB or MAPLE whenever possible. 


Show that the matrix 38 Given 
2 3 1 
A= 5 6 
2 3 A=/3 1 2 
F : ae ‘ 12 3 
satisfies its own characteristic equation. 


compute A’ and, using the Cayley—Hamilton 
Given theorem, compute 


; 1 A’ — 30° + At + 3A — 2A? +31 


39 Evaluate e*’ for 
use the Cayley—Hamilton theorem to evaluate 
A= eo b) A= " 
(a) A (b) Ac) A (a) A= (b) A= 


The characteristic equation of an n x n matrix A is 


40 Given 
K+ Cy 1H! + Cy, oh? +...4 04+ c= 0 
2 0 0 
so, by the Cayley—Hamilton theorem, A-2£lo 11 
5 2 
A’+c, A’'+c¢, A"? +...4+¢ At c1=0 00 1 
If A is non-singular then every eigenvalue is show that 
non-zero, sO Cy # 0 and 
0 0 0 
1 n n-l : = 4 4 2, 
l=-—(A'+c¢,_A"'+...4+¢,A) snA=-A-—=A=/|0 1 0 
ni ~ % 001 


which on multiplying throughout by A! gives 


A? =-L ars A+... te8) 44) EE Given 


n-l 


Co 
2 
(a) Using (1.44) find the inverse of the matrix A= et 2t=3 
i. F 5-t f-t+3 
A= evaluate 
1 2 
dA ; 
(b) Show that the characteristic equation of the (a) rr (b) | Adt 
matrix 1 


42 Given 


A- +l t-1 
5 0 


evaluate A’ and show that 


is 
B-3# -7TA-11=0 
dA 


Evaluate A’ and, using (1.44), determine A’. EA) #2A rn 
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Singular value decomposition 


Example 1.33 


Solution 


So far we have been concerned mainly with square matrices, dealing in particular with 
the inverse matrix, the eigenvalue problem and reduction to canonical form. In this 
section we consider analogous results for non-square (or rectangular) matrices, all of 
which have important applications in engineering. 

First we review some definitions associated with non-square matrices: 


(a) A non-square m x n matrix 
A=(a,),i=1,2,...,mj=1,2...,n 


is said to be diagonal if all the i, 7 entries are zero except possibly for i = 7. For 


example: 
[2 0 
0 3 is a diagonal 3 x 2 matrix 
19 0 
whilst 
[2 0 0 eens : 
is a diagonal 2 x 3 matrix 
iO 3 


(b) The row rank of a m X n matrix A denotes the maximum number of linearly 
independent rows of A, whilst the column rank of A denotes the maximum 
number of linearly independent columns of A. It turns out that these are the same 
and referred to simply as the rank of the matrix A and denoted by r = rank(A). It 
follows that r is less than, or equal to, the minimum of m and n. The matrix A is 
said to be of full-rank if r equals the minimum of m and n. 


For the 3 x 4 matrix 


12 3 4 
A=|/3 4 7 10 
2 ‘Lh 3 3 


confirm that row rank (A) = column rank (A). 


Following the process outlined in Section 1.2.6 we reduce the matrix to row (column) 
echelon form using row (column) elementary operations. 


(a) Row rank: using elementary row operations 


12 3 4 
3 4 7 10 
2 1 3 = #«=5 


| row 2—3 x row l, row 3 —2 x row 1 


www.2Ofile.org 


1.8 SINGULAR VALUE DECOMPOSITION 67 


1 2 3 +4 
0 -2 -2 -2 
0 -3 -3 -3 


| multiply row 2 by -} 


1 2 3 4 
0 1 1 1 
0 -3 -3 -3 


| row 3+3 x row 2 


12 3 4 
0 1 1 1 
0 0 
which is in row echelon form and indicating that 
row rank (A) = 2 
(b) Column rank: using elementary column operations 


23 4 
4 7 10 
1 3 5 


NO Woe 


| col2 — 2 x coll, col3 — 3 x coll, col4 — 4 x coll 


1 0 O 0 
Bo °=2. %=2. = 2 
2 -3 “3-3 


| col3 — col2, col4 — col2 


1 0 0 
3 -2 0 0 
2 3 0 0 


which is in column echelon form and indicating that 
column rank (A) = 2 

confirming that 
rank(A) = row rank (A) = column rank (A) = 2 


Note that in this case the matrix A is not of full rank. 
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1.8.1 


Singular values 


For am X n matrix A the transposed matrix A‘ has dimension n x m so that the product 
AAT is a square matrix of dimension m x m. This product is also a symmetric matrix 
since 


(AA’)' a (A')'(A’) = AA' 


It follows from Section 1.4.7 that the m x m matrix AA‘ has a full set of m linearly 
independent eigenvectors u,, u,..., u,, that are mutually orthogonal, and which 
can be normalized to give the orthogonal normalized set (or orthonormal set) of 
eigenvectors 


A A A 


Uj, Uy,..., Uy, 


with a/a, = 6, (i, 7 = 1, 2,..., m), where 6, is the Kronecker delta defined in 
Section 1.3.2. 

(Reminder: As indicated in Section 1.4.2 normalized eigenvectors are uniquely 
determined up to a scale factor of +1.) We then define the m x m orthogonal matrix U 
as a matrix having these normalized set of eigenvectors as its columns: 


O=([fi, i,,...,a 


with O'0 = 0U' =1 
Let A,,A,...,4 


(AA')i, =A,f,, i=1,2,...,m 


(1.45) 


m 


i Such a matrix is also called a unitary matrix. 
be the corresponding eigenvalues of AA' so that 


Considering the square of the length, or norm, of the vector Aa, then from orthogonality 


| Aai,| = (Aii;)'(Aii;) = ii) (A'Aa)) = aj Ajit; i A; 


(Note: the notation || Ad, |? is also frequently used.) Since |Aii,;|* = 0 it follows that the 
eigenvalues A,(i = 1, 2,..., m) of the matrix AA’ are all non-negative and so can be 
written in the form 


A,=06),i=1,2,...,m 
It is also assumed that they are arranged in a non-increasing order so that 
Oj, 2o;=...2=0720 


Some of these eigenvalues may be zero. The number of non-zero values (accounting 
for multiplicity) is equal to r the rank of the matrix A. Thus, if rank(A) = r then the 
matrix AA’ has eigenvalues 

Oj 2o;=...20°>0witho,,=...=02=0 


The positive square roots of the non-zero eigenvalues of the matrix AA’ are called the 
singular values of the matrix A and play a similar role in general matrix theory that 
eigenvalues play in the theory of square matrices. If the matrix A has rank r then it has 
r singular values 


0, 20,2...20,>0 


In practice determining the singular values of a non-square matrix provides a means of 
determining the rank of the matrix. 
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Example 1.34 For the matrix 


3 
A=|1 3 
1 


(a) Determine the eigenvalues and corresponding eigenvectors of the matrix AA". 


(b) Normalize the eigenvectors to obtain the corresponding orthogonal matrix U and 
confirm that UU" =I. 


(c) What are the singular values of A? 


(d) What is the rank of A? 


Scale a 10 0 
Solution (2) AA'=|1 3 k |- 0 10 4 
1 4 2 


(Note that AAT is a symmetric matrix.) 
The eigenvalues of AA" are given by the solutions of the equation 


10-2 6 b 
|AA'—-Al|=| 0 10-A 4 |=0 


which reduces to 
(12 —A)10 - A)A=0 
giving the eigenvalues as 


A, = 12, A, = 10, A,=0 


Solving the homogeneous equations 
(AAT — AJ)u; = 0 
gives the corresponding eigenvectors as: 
uw=[1 2 1), m=[2 -1 OJ", w,=[1 2 —ST' 


(b) The corresponding normalized eigenvectors are: 


T T T 
“=|i 2 24 Mm=|;2 2 io|l 2% 3 
: E ye y6| ? 2 5 O) > 4s 730 ©)30~=—\30 


giving the corresponding orthogonal matrix 


cAIN. 


al- 


0.04082 0.8944 0.1826 
=10.8165 -0.4472 0.3651 
0.4082 0.0000 -0.9129 


AG 
al 
o 


aS 
ll 
= 
iss 
i) 
i~ 
al 
ll 
‘ale 
al 
gly 


|- 
So 
|b 


a 


www.2Ofile.org 


70 MATRIX ANALYSIS 


By direct multiplication 


ye vs 30 || y6 6 6 1 0 0 
vyyt_| 2 -1 2 el _ 
UU’ = 4 os wills 7 O;=|0 1 0 
L g =3 1 2. <5 00 1 
16 30 || 30 30 30 


confirming that UOT =1. 


(c) The singular values of A are the square roots of the non-zero eigenvalues of AA". 
Thus the singular values of A are o, = {12 and o, = \10. 


(d) The rank of A is equal to the number of singular values giving rank (A) = 2. This 
can be confirmed by reducing A to echelon form. 


Likewise, for a m X n matrix A the product A‘A is a square n X n symmetric matrix, 


having a full set of n orthogonal normalized eigenvectors 0;, 05,..., 6, which form the 
columns of the n x n orthogonal matrix V: 
V=[6, 0)...8, (1.46) 
and having corresponding non-negative eigenvalues L,, L,,..., UM, with 
UW2w]...2u,29 (1.47) 


Again the number of non-zero eigenvalues equals r, the rank of A, so that the product 
A'A has eigenvalues 


MW 2=wh=...2u,>0withy, =...=",=0 
Thus 

AAG, = 0, “W;>0@=1,2,...,n (1.48) 
Premultiplying by A gives 

(AA')(Ad,) = u(Ad;) 
so that yz, and (Ad) are an eigenvalue and eigenvector pair of the matrix AA’; indicating 
that the non-zero eigenvalues of the product AA’ are the same as the non-zero eigen- 


values of the product A'A. Thus if A is of rank r then the eigenvalues (1.47) of the 
product ATA may be written as 


CO, 1H 1,2 ccey Fh 
Melo ver 
In general the vector (A@;,) is not a unit vector so 
Ad, = kii; (1.49) 
and we need to show that k = o,. Taking the norm of (Ad,) gives 
|Aé, = (Aé,)'(Ad,) 


= 6; Ub; from (1.48) 
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giving 
|Ad,| =k= 0; 
It follows from (1.49) that 
0,f;,i=1,2,...,7 
{ (1.50) 
OQ,i=rt+l1,...,m 


Clearly the singular values of A may be determined by evaluating the eigenvalues of 
the product AA! or the product A‘A. The eigenvectors ii, ii,,..., i, of the product 
AA’ (that is the columns of 0) are called the left singular vectors of A and the eigen- 
vectors 0,, ),..., 6, of the product A‘A (that is columns of V) are called the right 
singular vectors of A. 


For the matrix 


3 = 
A=|1 3 
1 1 
(a) Determine the eigenvalues and corresponding eigenvectors of the product A'A. 
(b) Normalize the eigenvectors to obtain the orthogonal matrix V. 


(c) What are the singular values of A? 


_{ll 1 
1 ou 


The eigenvalues of A’A are given by the solutions of the equation 


3 

3 1 | 
(a) an | if 
1 


ite: 9 
1 ll-u 


[ATA — l= 


which reduces to 
(u— 12)(u— 10) =0 

giving the eigenvalues as 
HM, = 12, wu, = 10 

Solving the homogeneous equations 
(A'A- uN) v,=0 

gives the corresponding eigenvectors as 


y=! 17, »%=[1 -1]' 
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1.8.2 


(b) The corresponding normalized eigenvectors are: 


1 iy" 1 i]" 
M=|5 sl -Mmals Fs 
: E S| >A E 7] 


giving the orthogonal matrix 


-1| |0,.7071  -0.7071 


ae 
y2 V 


p.|2 2 Ri i 
-1 


(c) The singular values of A are the square roots of the non-zero eigenvalues of A'A. 
Thus the singular values of A are: 


O, = i, = \12 = 3.4641 and o, = /10=3.1623 


in agreement with the values obtained in Example 1.34. 


Singular value decomposition (SVD) 


For an m X n matrix A of rank r the m equations (1.50) can be written in the partitioned 
form 


Alb,0,...6,|6.,;...6,)=[aa,...%,| a... 4, JE (1.51) 


where the matrix & has the form 


oO, O - 0140 0; A 
Oe a |G ee 6 
z=|0 0 o, v 
0 0 0 A 
m-r 
| 0 0 v 
< > << > 
ti ni=-fr 
where 0), O>,..., 0, are the singular values of A. More precisely (1.51) may be 
written as 
AV = U0 


Using the orthogonality property VV" = I leads to the result 
A=UxV" (1.52) 


Such a decomposition (or factorization) of a non-square matrix A is called the 
singular value decomposition of A, commonly abbreviated as SVD of A. It is 
analogous to the reduction to canonical (or diagonal) form of a square matrix developed 
in Section 1.6. 
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Solution 


Example 1.37 
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Find the SVD of the matrix 


and verify your answer. 


The associated matrices U and V and the singular values of A were determined in 
Examples 1.34 and 1.35 as: 


(5 (30 i ii 
fp | 2 -l v) fy |\2  \2 = 4 —j 
U=) i js pol P=| h) a= v2 and a, = y10 
1 -5 2 2 
16 0 30 


ale 
oe 
S 
‘S 
—) 
|- 
|= 


> 
Il 
lv 
al 
| 
i) 
S 


ts) 
— 
aS) 


|- 
SS 
|b, 
(=>) 
iS 
rs 
o 


Direct multiplication of the right hand side confirms 


3 
A=|1 3 
1 


The decomposition (1.50) can always be done. The non-zero diagonal elements of & 
are uniquely determined as the singular values of A. The matrices U and V are not 
unique and it is necessary to ensure that linear combinations of their columns satisfy 
(1.50). This applies when the matrices have repeated eigenvalues, as illustrated in 
Example 1.37. 


Find the SVD of the matrix 


oe °° co — 
oonN o 
oN CO OS 
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Solution AA’ = 


Koes a <> > 
Coon oOo 
oN CO SO 
So oa — 
oof CO 
of CO oO 
> ae) 


The product AA" has eigenvalues 1, = 4, A, = 4, A, = 1 and A, = 0. Normalized eigen- 
vectors corresponding to A, and A, are respectively 


a,=[1 0 0 OJ" and a =[0 0 0 17 


Various possibilities exist for the repeated eigenvalues A, = A, = 4. Two possible 
choices of normalized eigenvectors are 


a,=[0 1 0 OJ and a@=[0 0 1 of 
or 


a; = 50 1 1 OJ’ and #=4[0 1 -1 oy 


\2 


(Note that the eigenvectors #; and #3 are linear combinations of #, and a@,.) Likewise 


1 0 0 
AA'=/|0 2 0 
0 02 


oo fo 


0 
0 
2 
0 


co oc oOo = 
ooN CO 


and has eigenvalues u, = 4, U, = 4 and , = 1. The normalized eigenvector correspond- 
ing to the eigenvalue LU, = | is 


6,=[1 0 oy 


and two possible choices for the eigenvectors corresponding to the repeated eigenvalue 
H, =H, =4 are 


6,=[0 1 OJ" and 6,=[0 0 1] 
or 
bj=4[0 1 1" and 6f=4f0 1 -1]" 


The singular values of A are 0, = 2, 0, = 2 and 0; = | giving 


coo olUwN 
oonN oo 
oe SoS. So 


Considering the requirements (1.50) it is readily confirmed that 


Ado, = 0,i1,, Ab=», = 0,1, and Ad, = 0;i1, 
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so that 
0 0 1 0 . a4 
O-|) 9° 9%) mah=l1 0 0 
‘~]0 1 0.0 
0 1 O 
0 0 0 1 


reduces A to the SVD form A= U,ZV1. 
Also, it can be confirmed that 


Ai / = 0,ii{, Ab; = 0,15, Ab, = 0,ii, 


so that the matrix pair 


0 0 1 0 
: —s 0 0 1 
jan Pld ae 6 
U, 1 = 0 0 > V, \2 \2 
2 2B toi G 
2 2 
0 0 0 1 ‘ : 


reduces A to the SVD form 

A=0,xV! 
However, the corresponding columns of the matrix pair U,, V, do not satisfy conditions 
(1.50) and 

Az 0,xV" 


To ensure that conditions (1.50) are satisfied it is advisable to select the normalized 


eigenvectors 0, first and then determine the corresponding normalized eigenvectors i; 
directly from (1.50). 


Pseudo inverse 


In Section 1.2.5 we considered the solution of the system of simultaneous linear 
equation 
Ax=b (1.53) 


where A is the n x n square matrix of coefficients and x is the vector of unknowns. 
Here the number of equations is equal to the number of unknowns and a unique solution 


x=A'b (1.54) 
exists if and only if the matrix A is non-singular. 


There are situations when the matrix A is singular or a non-square m x n matrix. If 
the matrix A is am x n matrix then: 


e if m > n there are more equations than unknowns and this represents the over 
determined case; 


e ifm <n there are fewer equations than unknowns and this represents the under 
determined case. 
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Example 1.38 


Solution 


Clearly approximate solution vectors x are desirable in such cases. This can be achieved 
using the SVD form (1.52) of a m x n matrix A. Recognizing the orthogonality of U 
and V the following matrix A‘ is defined 


At=Vr°0" (1.55) 


where &* is the transpose of Z in which the singular values o; of A are replaced by their 
reciprocals. The matrix A’ is called the pseudo inverse (or generalized inverse) of the 
matrix A. It is also frequently referred to as the Moore—Penrose pseudo inverse of A. 
It exists for any matrix A including singular square matrices and non-square matrices. 
In the particular case when A is a square non-singular matrix A‘ = A™'. Since 


ro: 0 
A'A=|... 3... 
0 : O 
a solution of (1.53) is A‘Ax = A‘, that is 
x=A'b (1.56) 


This is the least squares solution of (1.53) in that it minimizes (Ax — b)'(Ax — b), the 
sum of the squares of the errors. 


Determine the pseudo inverse of the matrix 


3 v=! 
A=)1 3) 
1 1 


and confirm that ATA = I. 


From Example 1.36 the SVD of A is 


5 30 | | (12 O};fir 4 
Aw 2 = 2 ; V2 y2 
A=UxV'= \6 15 30 0 10 1 “| 
ae a 0 O0;b? 

\6 0 30 


The matrix &* is obtained by taking the transpose of % and inverting the non-zero 
diagonal elements, giving 


[to 
|H 


+ tlk o of] f & iy 4 5 
t+ Pye ft fv2 2] f2 2 =-1 = 
cg e iy el ¢ & |e : a 16 : 
v2 2 10 i. i] 5 


| 


w 
i) 
w 
Ss 
w 
S 
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Direct multiplication gives 


q <1 
ata-2[7 4 S]}, {4/60 Oo] _, 
116 S|), 0 60 


so that A‘ is a left inverse of A. However, A‘ cannot be a right inverse of A. 


We noted in the solution to Example 1.38 that whilst A‘ was a left inverse of A it was 
not a right inverse. Indeed a matrix with more rows than columns cannot have a right 
inverse, but it will have a left inverse if such an inverse exists. Likewise, a matrix with 
more columns than rows cannot have a left inverse, but will have a right inverse if such 
an inverse exists. 

There are other ways of computing the pseudo inverse, without having to use SVD. 
However, most are more restrictive in use and not so generally applicable as the SVD 
method. It has been shown that A‘ is a unique pseudo inverse of an m X n matrix A 
provided it satisfies the following four conditions: 


AA‘ is symmetric 

A‘A is symmetric (1.57) 
AA'‘A=A : 
A'AA'= A‘ 


For example, if an m x n matrix A is of full rank then the pseudo inverse may be 
calculated as follows: 


ifm >n then At =(ATAy'AT (1.58) 
ifm <n then At = A‘(AA’y! (1.58b) 


It is left as an exercise to confirm that these two forms satisfy conditions (1.57). 


(a) Without using SVD determine the pseudo inverse of the matrix 


3-1 
A=|1 3 
1 
(b) Find the least squares solution of the following systems of simultaneous linear 
equations 
(i) 3x-y=2 (ii) 3x-y=2 
x+3y=4 x+3y=2 
x+y=2 x+y=2 


and comment on the answers. 
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Solution (a) From the solution to Example 1.34 rank(A) = 2, so the matrix A is of full rank. 
Since in this case m > n we can use (1.58a) to determine the pseudo inverse as 


-1 
At=~atay'at= |!) St 4 
L. i |e a 4 


_i[/l -1]/ 3 1 1 
eet it) 3: 


1/17 4 5} _| 0.2833 0.0667 0.0833 
MHF 16 4 -0.1167 0.2667 0.0833 


in agreement with the result obtained in Example 1.38. 


(b) Both (i) and (ii) are examples of over determined (or over specified) sets of 
equations Ax = b with A being an m Xn matrix, m > n, b being an m-vector and 
x an n-vector of unknowns. Considering the augmented matrix (A:b) then: 


e ifrank(A:b) > rank(A) the equations are inconsistent and there is no solution 
(this is the most common situation for over specified sets of equations); 

e if rank(A:b) = rank(A) some of the equations are redundant and there is a 
solution containing n — rank(A) free parameters. 


(See Section 5.6 of Modern Engineering Mathematics.) 
Considering case (i) 


3-1 2 
A=/1 3},b=/4| and =" 
1 1 2 » 
3 -l 2 
rank(A:b) = rank | 1 3. 4) =2=rank(A) from (a). 
1 i 2 


Thus the equations are consistent and a unique solution exists. The least squares 
solution is 


2 
“| ayeaf!? 4 5/4) _]) 
y -7 16 S|},} [A 


which gives the unique solution x = y= 1. 
Considering case (ii) A and x are the same as in (i) and b = [2 2 2]" 


3 -1 2 
rank(A:b) =rank} 1 3 2) =3 > rank(A)=2 
1 1 2 
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Thus the equations are inconsistent and there is no unique solution. The least 
squares solution is 


2 
«| tye al)? 4 5 |= 4/3 
y oF M6: S|), 7 


ik _B a4 
giving x= 5 and y= 7. 


As indicated earlier, the least squares solution x = A‘b of the system of equations Ax = b 
is the solution that minimizes the square of the error vector r = (Ax — b); that is, mini- 
mizes (Ax — b)'(Ax — b). 

In practice, data associated with individual equations within the set may not be 
equally reliable; so more importance may be attached to some of the errors r,. To 
accommodate for this, a weighting factor (positive number) w, is given to the i” equa- 
tion (i= 1, 2,..., m) and the least squares solution is the solution that minimizes the 
square of the vector W(Ax — b), where W is the is the n x n diagonal matrix having 
the square roots \w; of the weighting factors as its diagonal entries; that is 


WW 0 ae 0 
wa| 0 We 0 
0 1 ar Wn 


The larger w; the closer the fit of the least squares solution to the i,, equation; the 
smaller w; the poorer the fit. Care over weighting must be taken when using least 
squares solution packages. Most times one would notice the heavy weighting, but in 
automated systems one probably would not notice. Exercise 49 serves to illustrate. 


In MATLAB the command 
svd (A) 

returns the singluar values of A in non-decreasing order; whilst the command 
[U,S,V]=svd (A) 


returns the diagonal matrix §' = and the two unitary matrices U= U and V=V such 
that A= USV". The commands: 


A=sym(A); 
svd (A) 


return the singular values of the matrix A in symbolic form. Symbolic singular vec- 
tors are not available. The command: 


pinv (A) 


returns the pseudo inverse of the matrix A using the SVD form of A. 
Using the matrix A of Examples 1.35, 1.36, 1.38 and 1.39 the commands 


Aes pig spi il; 
[U,S,V]=svd (A) 
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-0.4082 0.8944 -0.1826 
W=S—O S165 =O wae, =O. seal 
-0.4082 -0.0000 0.929 


3.4641 0 
S= 0 35 IGA 3 
0) 0) 


ao ee One Oi al 
=O. 7071 =0), 70711 
The additional command 
pinv (A) 
returns the pseudo inverse of A as 
0.29323 0.0667 0.0833 
=0 1167 O.2667 00833 
The commands: 
A=|3) sigil Bed sis 
a=sym(A); 
S=svd (A) 
return 
_ Bes) 
LO (Ly) 
In MAPLE the commands 
with (LinearAlgebra) : 
Ag=mencrsse (| L3,—il]|, (il, 31, ial, su es 
suc aSiincularvalues (A, culo |7U" , “SS”, wer I) s 
return 
=), A32 0.8944 a e25| 3.4641 


syvel=|—-O.8165 -0 4472 -O0.2365i\ > |3.1623)\, 
-0.4082 -0.0004 0.9125; |0.0000 


=O VOL 0. YO WL 
O.7O7L =O ,07071 


where the singular values are expressed as a vector. To output the values of U and 
Vt separately and to output the singular values as a matrix the following additional 
commands may be used 


Ws=sswell 1] § 
Wit seasvelll 3 lls 
SS:=matrix(3,2,(1,j3) -% if i=j then svd[2][ilelse 0 
fi);#output the singular values into a 3 2 matrix 
The further command 
Uecceves 
gives the output 


3.0000 =1 OWDO 
1.0000 3.0000 
1.000 1.000 
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confirming that we reproduce A. 
To obtain the pseudo inverse using MAPLE the normal matrix inverse command 
is used. Thus the commands 
with (LinearAlgebra) : 
ApeMaeei2e ( (LS,—1i], (1,3), ll, 1d). 
MatrixInverse (A) ; 


return 


i aye 
als 
ale 
Sie 


a|~ 
ol 
Sle 


in agreement with the answer obtained in Example 1.38. 


1.8.4 Exercises 


Use MATLAB or MAPLE to check your answers. 


Considering the matrix 46 Considering the matrix 
I> 22 8 4 1: = 
A=/3 4 7 10 A= | -2 Z 
2 A <5 7 D Sz 
(a) Determine row rank (A) and column rank (A). (a) What is the rank of A? 
(b) Is the matrix A of full rank? (b) Find the SVD of A. 
(a) Find the SVD form of the matrix (c) Find the pseudo inverse A‘ of A and confirm 
that AA'A = A and A‘AAt = AI, 
A= 4 11 4 (d) Find the least squares solution of the 
8 7 - simultaneous equations 


(b) Use SVD to determine the pseudo inverse A‘ of a a a Ee i a a 


the matrix A. Confirm that 4'A = 1. (e) Confirm the answer to (d) by minimizing the 


(c) Determine the pseudo inverse without using square of the error vector 


SVD. (Ax —b)whereb=[1 2 3]. 
Show that the matrix 47 — Considering the matrix 
1 1 3 = 
3 0 A=/|1 3 
A=|-2 1 1 
0 2 : é F 
12 (a) Use the pseudo inverse A‘ determined in 


Example 1.38 to find the least squares solution 


: : : $n for the simultaneous equations 
is of full rank. Without using SVD determine its a 


pseudo inverse 4‘ and confirm that A'A = I. 3x -—y=l,x+3y=2,x+y=3 
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(b) Confirm the answer to (a) by minimizing the (a) (i) 2x +y=3 (ii) 2x +y =3 
square of the error vector x+2y=3 x+2y=3 
+y=2 +y=3 
(Ax — b) whereb=[1 2 3]. oi eis 
(b) (i) 2x +y =3 (ii) 2x +y =3 
(c) By drawing the straight lines represented by the x+2y=3 x+2y=3 
equations illustrate your answer graphically. 10x + 10y = 20 10x + 10y = 30 

48 — Considering the matrix (c) @) 2x+y=3 (ii) 2x + y =3 

x+2y=3 x+2y=3 
1 0 2 100x + 100y = 200 100x + 100y = 300 

gu|o ft 1 Comment on your answers. 
=1 1 1 50 By representing the data in the matrix form Az = y, 
2 -1 2 where z = [m c]', use the pseudo inverse to find the 
values of m and c which provide the least squares fit 
(a) Show that A is of full rank. to the linear model y = mx + c for the following data. 
: ? ; 
(b) Determine the pseudo inverse A’. k 1 2 3 4 5 
(c) Show that the A‘ obtained satisfies the four Xp 0 1 2 2 4 
conditions (1.57). Vy 1 1 2 2 3 
49 Find the least squares solution of the following (Compare with Example 2.17 in Modern 


pairs of simultaneous linear equations. 


Engineering Mathematics.) 


State-space representation 


1.9.1 


In Section 10.11.2 of Modern Engineering Mathematics it was illustrated how the solu- 
tion of differential equation initial value problems of order n can be reduced to the 
solution of a set 7 of first-order differential equations, each with an initial condition. In 
this section we shall apply matrix techniques to obtain the solution of such systems. 


Single-input-single-output (SISO) systems 


First let us consider the single-input—single-output (SISO) system characterized by 
the nth-order linear differential equation 


n n-1 
aitea, SP %+...4a, 2+ ay = uly) (1.59) 
an ae di 
where the coefficients a; (i= 0, 1, ..., ) are constants with a, # 0 and it is assumed 
that the initial conditions y(0), y“?(0), ... , (0) are known. 
We introduce the n variables x,(t), x,(t), ... , x,(t) defined by 
x(t) = (0) 


x(t) = 2 =%4(0) 


2 


xs(Q = 2 = 2,0 
dt 
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n-2 


x,1() = 4-2 = 5, 9(0) 
dt 


7 dq’! _ 
X,(t) = = X,-1(t) 


n-1 
where, as usual, a dot denotes differentiation with respect to time ¢. Then, by substitut- 
ing in (1.59), we have 
Ay Xn + Ay Xq + Ay Xp) $2 FAX) + AX, = u(t) 
giving 
Sul y An-2 ay a 


1 
Kia eg a ho SS i Se 
ay ay, ay, ay, an 


Xn = 


Thus, we can represent (1.59) as a system of n simultaneous first-order differential 
equations 


X= X2 
X= 5 
Xn-1 = Xn 
: a a om 1 
X,=—2x,-txy- 2... - ey, + -u 
ay an ay an 


which may be written as the vector—matrix differential equation 


x, 0 1 OM sed 0 0 x 0 

Xy 0 0 1 eid 0 0 Xo 0 
= ||\2 - |+{- |u(t) (1.60) 

Xn 0 0 0 0 1 Xn 0 

749 74, —ay T4n-2 7 An-1 1 

Xn eo) aa are aunt aaa arama Xn — 

ay ay ay, ay, ay, a, 
(Note: Clearly x,, x5, ..., x, and uw are functions of ¢ and strictly should be written as 
x,(f), x(t), ... , x,(2) and u(t). For the sake of convenience and notational simplicity the 


argument (f) is frequently omitted when the context is clear.) 
Equation (1.60) may be written in the more concise form 


x= Ax + bu (1.61a) 


The vector x(f) is called the system state vector, and it contains all the information that 
one needs to know about the behaviour of the system. Its components are the 7 state 
variables x,, x5, ..., X,, which may be considered as representing a set of coordinate 
axes in the n-dimensional coordinate space over which x(f) ranges. This is referred to 
as the state space, and as time increases the state vector x(f) will describe a locus in this 
space called a trajectory. In two dimensions the state space reduces to the phase plane. 
The matrix A is called the system matrix and the particular form adopted in (1.60) is 
known as the companion form, which is widely adopted in practice. Equation (1.61a) 
is referred to as the system state equation. 
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The output, or response, of the system determined by (1.59) is given by y, which in 
terms of the state variables is determined by x,. Thus 


x} 
X2 
yal! 0 O]] . 
xX; 
or, more concisely, 
yecx (1.61b) 
wherec=[1 0 ... OJ. 


A distinct advantage of the vector—matrix approach is that it is applicable to 
multivariable (that is, multi-input—multi-output MIMO) systems, dealt with in Section 
1.9.2. In such cases it is particularly important to distinguish between the system state 
variables and the system outputs, which, in general, are linear combinations of the 
state variables. 

Together the pair of equations (1.61a,b) in the form 


X= Ax + bu (1.62a) 
yee'x (1.62b) 


constitute the dynamic equations of the system and are commonly referred to as the 
state-space model representation of the system. Such a representation forms the basis 
of the so-called ‘modern approach’ to the analysis and design of control systems in 
engineering. An obvious advantage of adopting the vector—matrix representation (1.62) 
is the compactness of the notation. 

More generally the output y could be a linear combination of both the state and input, 
so that the more general form of the system dynamic equations (1.62) is 


x= Ax + bu (1.63a) 
y=c'xtdu (1.63b) 
Comment 
It is important to realize that the choice of state variables x,, x5, ..., x, is not unique. 


For example, for the system represented by (1.59) we could also take 


ites — An-2 a _ao| 1 
ay ay ay n ay 0 
1 0 0 0 0 0 
A= 5 b= 5 c= (1.64) 
0 1 0 : 
1 
ie a 1 0 | | 0 | 
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Obtain a state-space representation of the system characterized by the third-order 
differential equation 


3 2 
dy 4 3404 2 _4y =e" (1.65) 
dt dt dt 
Writing 
2 
X=, ma DSH, x, = =x, 
dt dt 


we have, from (1.65), 


i= dy - ay-24 342464 = 4x, — 2x, — 3x, + 7 
dt dt dt 
Thus the corresponding state equation is 
x] fo 1 ol[m] fo 
X1=10 0 I1)|ml]+]ole’ 
x; 4 -2 -3||x;, 1 


These two equations then constitute the state-space representation of the system. 


We now proceed to consider the more general SISO system characterized by the 
differential equation 


n n-1 m 
hae d Ye tay = bb + 2. + Bqu (m <n) (1.66) 
dr’ dr” dt” 


in which the input involves derivative terms. Again there are various ways of representing 
(1.66) in the state-space form, depending on the choice of the state variables. As an illus- 
tration, we shall consider one possible approach, introducing others in the exercises. 

We define A and b as in (1.60); that is, we take A to be the companion matrix of the 
left-hand side of (1.66), giving 


0 1 0 0 0 
0 0 1 
A= : 
0 0 0 0 1 
4) ~a, —az TEn-2 ~An-1 
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Example 1.41 


Solution 


Figure 1.3 

Block diagram for the 
state-space model of 
Example 1.41. 


and we take b=[0 0 ... O 1)’. In order to achieve the desired response, the 
vector c is then chosen to be 


c=[b, b, ... by O ... OF (1.67) 


It is left as an exercise to confirm that this choice is appropriate (see also Section 5.7.1). 


Obtain the state-space model for the system characterized by the differential equation 
model 


3 2 2; 
dy 4 684 Wa ay = 5d My (1.68) 
dt dt dt dr dt 


Taking A to be the companion matrix of the left-hand side in (1.68) 


0 1 0 
A=| 0 0 1! and b=[0 0 1]" 
-—3 -ll -6 


we have, from (1.67), 
c=[l1 1 5]' 
Then from (1.62) the state-space model becomes 


X=Ax+bu, y=c'x 


This model structure may be depicted by the block diagram of Figure 1.3. It provides 
an ideal model for simulation studies, with the state variables being the outputs of the 
various integrators involved. 


A distinct advantage of this approach to obtaining the state-space model is that A, b 
and ¢ are readily written down. A possible disadvantage in some applications is that the 
output y itself is not a state variable. An approach in which y is a state variable is 
developed in Exercise 56, Section 5.7.2. In practice, it is also fairly common to choose 
the state variables from a physical consideration, as is illustrated in Example 1.42. 


www.2Ofile.org 


1.9.2 


Example 1.42 


Figure 1.4 
Parallel circuit of 
Example 1.42. 


Solution 
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Multi-input-multi-output (MIMO) systems 


Many practical systems are multivariable in nature, being characterized by having more 
than one input and/or more than one output. In general terms, the state-space model is 
similar to that in (1.63) for SISO systems, except that the input is now a vector u(f) as 
is the output y(). Thus the more general form, corresponding to (1.63), of the state- 
space model representation of an nmth-order multi-input—multi-output (MIMO) system 
subject to r inputs and / outputs is 


¢ = Ax+Bu (1.69) 
y=Cx+Du (1.69b) 


where x is the n-state vector, u is the r-input vector, y is the /-output vector, A is the 
nXn system matrix, B is the n x r control (or input) matrix, and C and D are respect- 
ively / Xn and/ x r output matrices. 


Obtain the state-space model representation characterizing the two-input—one-output 
parallel network shown in Figure 1.4 in the form 


X=Ax+Bu, y=c'x+d'u 


where the elements x,, x5, x; of x and u,, wu, of u are as indicated in the figure, and the 
output y is the voltage drop across the inductor L, (vc denotes the voltage drop across 
the capacitor C). 


Applying Kirchhoff’s second law (see Section 5.4.1) to each of the two loops in turn 
gives 


ES ieee (1.70) 
dt 
4 

is rr +c => (1.71) 


The voltage drop v- across the capacitor C is given by 
F Le. ai 
Uc= rake + iy) (1.72) 


The output y, being the voltage drop across the inductor L,, is given by 


di, 


— we 
Ne 
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Sl 


By2 


which, using (1.70), gives 


y=-Rii — Uc t+ e 


Writing x, = 1,, X) = bh, X3 = Ve, Uy = | 
representation as 
r 7 TR alia 
rt Pabiceci | 0 easy 
x iB is xX 
a| oo a SgaS 
L, 
: 1 1 
=| |G ¢ © || 
mal 
y=[-Ri 0 -1))/%) +11 
x3 


which is of the required form 
x=Ax+ Bu 


y=e'xt+d'u 


1.9.3 Exercises 


Obtain the state-space forms of the differential 
equations 


dy, ,d@y,<d 
(a) 44524 5% 4 4yaeu(s) 
dt dt dt 
4 2 
(bo) $2424 4B = sur) 
dt dt dt 


using the companion form of the system matrix in 
each case. 


Obtain the state-space form of the differential 
equation models 


@ SPs 68 4 5Vary = fu, du 5, 
d* df dt df? dt 
ty) Ps 4h 3 dy du soy 


dr dt df 


using the companion form of the system matrix in 
each case. 


Bs) 


54 


(1.73) 


and u, = e,, (1.70)—(1.73) give the state-space 


Lt 

Ly 

ee lh 
L, Uy 

0 0 


Obtain the state-space model of the single-input— 
single-output network system of Figure 1.5 in the 
form x = Ax + bu, y = c'x, where u, y and the 
elements x), x», x; of x are as indicated. 


UG = 43 


Figure 1.5 Network of Exercise 53. 


The mass—spring—damper system of Figure 1.6 
models the suspension system of a quarter-car. 
Obtain a state-space model in which the output 
represents the body mass vertical movement y 

and the input represents the tyre vertical movement 
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u(t) due to the road surface. All displacements are 55 
measured from equilibrium positions. 
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Obtain the state-space model, in the form 

x = Ax + bu, y= Cx + du of the one-input— 
two-output network illustrated in Figure 1.7. The 
elements x,, x, of the state vector x and y,, y2 of 
the output vector y are as indicated. If R, = 1kQ, 
R, = 5kQ, R, = R= 3kQ, C,= C,= 1 pF 
calculate the eigenvalues of the system 

matrix A. 

absorber 


Axle and 
wheel mass 


Road profile 


Figure 1.6 Quarter-car Suspension model of 


Exercise 54. 


Figure 1.7 Network of Exercise 55. 


Solution of the state equation 


1.10.1 


In this section we are concerned with seeking the solution of the state equation 


x=Ax+ Bu (1.74) 


given the value of x at some initial time f, to be x). Having obtained the solution of this 
state equation, a system response y may then be readily written down from the linear 
transformation (1.69b). As mentioned in Section 1.9.1, an obvious advantage of adopt- 
ing the vector—matrix notation of (1.74) is its compactness. In this section we shall see 
that another distinct advantage is that (1.74) behaves very much like the corresponding 
first-order scalar differential equation 


dx 


—==axt bu 
dt : 


X(t) = Xp (1.75) 
Direct form of the solution 
Before considering the mth-order system represented by (1.74), let us first briefly review the 


solution of (1.75). When the input w is zero, (1.75) reduces to the homogeneous equation 


ds ae (1.76) 


which, by separation of variables, 


| ial a dt 
os i 


Xo 0 
gives 


In x — In x) = a(t — th) 
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leading to the solution 
x=x, ec (1.77) 


for the unforced system. 
If we consider the nonhomogeneous equation (1.75) directly, a solution can be 
obtained by first multiplying throughout by the integrating factor e to obtain 


(2 - ax] =e“ bu(t) 


or 


f(x) =e" butt) 


which on integration gives 


t 
ev x-e "x)= | ee" bul t) dt 
% 
leading to the solution 
t 
x(t) = x5 4 | e” bu(t) dt (1.78) 
% 
The first term of the solution, which corresponds to the solution of the unforced system, 
is a complementary function, while the convolution integral constituting the second 
term, which is dependent on the forcing function u(f), is a particular integral. 
Returning to (1.74), we first consider the unforced homogeneous system 
X=Ax, X(t) =X (1.79) 


which represents the situation when the system is ‘relaxing’ from an initial state. 
The solution is completely analogous to the solution (1.77) of the scalar equation (1.76), 

and is of the form 

x= ey, (1.80) 
It is readily shown that this is a solution of (1.79). Using (1.36), differentiation of (1.80) 
gives 

X= Ae“ x, =Ax 
so that (1.79) is satisfied. Also, from (1.80), 


A(ty-%9) 


X(ty) =e Xo = Ixy = Xo 


using e° = I. Thus, since (1.80) satisfies the differential equation and the initial condi- 
tions, it represents the unique solution of (1.79). 

Likewise, the nonhomogeneous equation (1.74) may be solved in an analogous man- 
ner to that used for solving (1.75). Premultiplying (1.74) throughout by e*’, we obtain 


e* (x — Ax) = e*Bu(t) 
or using (1.36), 


fe™x) =e Bu(t) 
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Figure 1.8 

(a) Transition 
matrix (ft, f). 
(b) The transition 
property. 

(c) The inverse 
Dt, to). 
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Integration then gives 
t 
e'x(1) - 8x, = | e *’Bu( 1) dt 
% 


leading to the solution 


A(t-19) 


x(th=e "xo+ | Bult) dt (1.81) 


0 


This is analogous to the solution given in (1.78) for the scalar equation (1.75). Again it 
contains two terms: one dependent on the initial state and corresponding to the solution 
of the unforced system, and one a convolution integral arising from the input. Having 
obtained the solution of the state equation, the system output y(f) is then readily obtained 
from equation (1.69b). 


The transition matrix 
The matrix exponential ce) is referred to as the fundamental or transition matrix 
and is frequently denoted by ®(t, t)), so that (1.80) is written as 

x(t) = DE, ty) Xp (1.82) 


This is an important matrix, which can be used to characterize a linear system, and in 
the absence of any input it maps a given state x, at any time f, to the state x(t) at any 
time ¢, as illustrated in Figure 1.8(a). 


x(t) X(15) x(t) 


Bit, ty) Pity, t3) P(t, 11) P(t, ty) 
x(t)) 


Dt, ty) @-l(t, ty) 
X(fy) X(t) X(ty) 
(a) (b) (c) 


Using the properties of the exponential matrix given in Section 1.7, certain properties 
of the transition matrix may be deduced. From 
A(t, +1) Ai, At, 
e ete? 


it follows that ®(¢, f)) satisfies the transition property 
Dt, t) = P(t, t) P(t, bh) (1.83) 
for any fo, tf, and ¢,, as illustrated in Figure 1.8(b). From 
At At | 


ee 


it follows that the inverse ®'(, f)) of the transition matrix is obtained by negating time, 
so that 


D'(t, {)) = D(-t, ty) = Pty, 1) (1.84) 


for any ft, and ¢, as illustrated in Figure 1.8(c). 
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1.10.3 


Example 1.43 


Solution 


Evaluating the transition matrix 


Since, when dealing with time-invariant systems, there is no loss of generality in taking 
ty = 0, we shall, for convenience, consider the evaluation of the transition matrix 


D(t) = Dt, 0) =e 

Clearly, methods of evaluating this are readily applicable to the evaluation of 
@(1, t) = e869 

Indeed, since A is a constant matrix, 
@(t, t) = B(t— 1, 0) 


so, having obtained ®(t), we can write down ®(t, T) by simply replacing ¢ by t — T. 
Since A is a constant matrix the methods discussed in Section 1.7 are applicable for 
evaluating the transition matrix. From (1.34a), 


“= a ,(t)l+ a,(H)A+ a,()0A+...4+0,_(HA"! (1.85a) 
where, using (1.34b), the a(t) (i= 0, 1,...,— 1) are obtained by solving simul- 
taneously the n equations 

ef! =a,(t)+ aOA,t+ a(O8+...+,.()A7" (1.85b) 


where A, (j = 1, 2, ..., m) are the eigenvalues of A. As in Section 1.7, if A has 
repeated eigenvalues then derivatives of e“’, with respect to A, will have to be used. 


A system is characterized by the state equation 


C(t = t 

X(t) aL o x(t) n 1 u(t) 

X,(t) 1 —3)]| x(t) 1 
Given that the input is the unit step function 


0 (t<0) 


u(t) = H(t) = { or 


and initially 
x,(0) = x,(0) = 1 


deduce the state x(t) =[x,(t) x,(t)]’ of the system at subsequent time ¢. 


From (1.81), the solution is given by 


x(t) = e™ x(0) + | en bu(t) dt (1.86) 
0 
where 
-1 0 
A= , sea af 
. Ey fl 1] 
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Since A is a 2 x 2 matrix, it follows from (1.85a) that 
W=a,()l+a(ynA 

The eigenvalues of A are A, =—1 and 1, = —3, so, using (1.85), we have 
a(t)=;GBet-e*), a,() =} (e*-e*) 


giving 


Thus the first term in (1.86) becomes 


-t 
t 0 
eM x(0) = e 1 _ e 


ie" = eo} e! l he" of o) 


and the second term is 


" t -(t-1) 
= e 0 1 
| eM ‘ounae= | ae ldt 
e 


1 -(t-+t) —3(t-T) 
0 0 3(e —e ) 


t eH 
= 1, (1), -3(t-7) dt 
o | se +e ) 


ie t 
—(t-1) 
e 


1 —(i-T) 1 .—3(t-7) 
3(e +3e€ ) 0 


e° e- 
fie te)] | eet +e) 
L l-e" 
a 2 = let Le™! 


That is, 


at=1,. a= 2ele" 
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Using the Symbolic Math Toolbox in MATLAB the transition matrix e” is gener- 
ated by the sequence of commands 


syms t 
A=[specify]; 
A=sym(A); 


E=expm(t*A) ; 
pretty (E) 


Confirm this using the matrix A = [—1 0; 1 —3] of Example 1.43. 
In MAPLE e*’ is returned by the commands: 


with (LinearAlgebra) : 
Ap=Neiem<( | \=L, Ol, il,=s ys 
MatrixExponential (A,t); 


1.10.4 Exercises 
Check your answers using MATLAB or MAPLE whenever possible. 


56 = Obtain the transition matrix @(f) of the system 59 Find the solution of 


x= Ax : 
~ |*1 
where =") 


-|° |e + [2 a (t = 0) 
-6 -5||x, 6 


where u(t) = 2 and x(0)=[1 —l]’. 


Verify that ®(t) has the following properties: 

(a) &(0)=1; 

(b) ®(t,) = O(,- 1) O(4,); 60 
(c) ®'(t) = B(-?). 


Using (1.81), find the response for ¢t = 0 of the 
system 


Xp =H, + 2u 


57 ~~ Writing x, = y and x, = dy/dt express the differential Pate 


equation 
# d to an input u(¢) = e“ and subject to the initial 
a +2 7 +y=0 conditions x,(0) = 0, x,(0) = 1. 
2 


61 A system is governed by the vector—matrix 
differential equation 


wne/) tor |} eo (t = 0) 
2 1 i - 4 


in the vector—matrix form x = Ax, x =[x, x]". 
Obtain the transition matrix and hence solve the 
differential equation given that y = dy/dt = 1 when 
t= 0. Confirm your answer by direct solution of the 
second-order differential equation. 


58 


Solve 


subject tox(0)=[1  1]’. 


where x(t) and u(t) are respectively the state 

and input vectors of the system. Determine 

the transition matrix of this system, and hence 
obtain an explicit expression for x(t) for the input 
u(t)=[4 3]' and subject to the initial condition 
x(0)=[1 2]'. 
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Spectral representation of response 
We first consider the unforced system 
x(t) = Ax(t) (1.87) 


with the initial state x(f)) at time f) given, and assume that the matrix A has as distinct 
eigenvalues A; (i= 1, 2,..., ) corresponding to n linearly independent eigenvectors 
e,(i=1,2,...,n). Since the n eigenvectors are linearly independent, they may be used 
as a basis for the n-dimensional state space, so that the system state x(t) may be written 
as a linear combination in the form 


x(t) =c(He, +...+¢,(He, (1.88) 


where, since the eigenvectors are constant, the time-varying nature of x(f) is reflected 
in the coefficients c(t). Substituting (1.88) into (1.87) gives 


é(te,+...+é,(He, = Alc (tHe, +...+c,(te,] (1.89) 
Since (/,, e,) are spectral pairs (that is, eigenvalue—eigenvector pairs) for the matrix A, 
Ae,=Aje, (i=1,2,...,2) 
(1.89) may be written as 
[é,(t) — Aye (t)Je,;+... + [é,(t) — ,c,(t)]e, = 0 (1.90) 


Because the eigenvectors e; are linearly independent, it follows from (1.90) that the 
system (1.87) is completely represented by the set of uncoupled differential equations 


é(t)-A,c(t)=0 (G@=1,2,...,n) (1.91) 
with solutions of the form 


A (t-t9) 


c(t) =e Ci( to) 


Then, using (1.88), the system response is 


x(1) = ito) ene (1.92) 
i=1 


Using the given information about the initial state, 


n 


X(to) = ¥ eilto) e; (1.93) 
i=l 
so that the constants c;(f)) may be found from the given initial state using the reciprocal 
basis vectors r; (i= 1, 2,..., n) defined by 
r= 6; 


where 6,,is the Kronecker delta. Taking the scalar product of both sides of (1.93) with 
r,, we have 


F_X(t) = Yi Cito) rre: = Cu(to) (k= 1,2,...5 0) 


i=l 
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Example 1.44 


Solution 


which on substituting in (1.92) gives the system response 


x(t) = MS 1: X(t) ee, (1.94) 


i=1 


which is referred to as the spectral or modal form of the response. The terms 
r, X(to) e 0) 9 are called the modes of the system. Thus, provided that the system 


matrix A has n linearly independent eigenvectors, this approach has the advantage of 
enabling us to break down the general system response into the sum of its simple modal 
responses. The amount of excitation of each mode, represented by r/x(t)), is dependent 
only on the initial conditions, so if, for example, the initial state x(t) is parallel to the 
ith eigenvector e, then only the ith mode will be excited. 

It should be noted that if a pair of eigenvalues 1,, 2, are complex conjugates then 


. i A, (t= Ax(t- 
the modes associated with e“!” and e“2”’ cannot be separated from each other. The 


combined motion takes place in a plane determined by the corresponding eigenvectors 
e, and e, and is oscillatory. 

By retaining only the dominant modes, the spectral representation may be used to 
approximate high-order systems by lower-order ones. 


Obtain in spectral form the response of the second-order system 


art leh “bl 


and sketch the trajectory. 


The eigenvalues of the matrix 


“fs 


are determined by 
|A-Al|=7+424+3=0 


that is, 


with corresponding eigenvectors 
a=) Al, @=[1 =i)" 
Denoting the reciprocal basis vectors by 
n= un rol’, r=[r Ty)" 
and using the relationships 


rie= 6; (i,j = 1, 2) 
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Trajectory for 
Example 1.44. 
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we have 

re=rytry=1, re, =", — 1, =0 
giving 

hi= 5: rue=ss n=[ yr 
and 

TO, = 1p, + My = 0,~ Ne) = 1x, —Ty = 1 
giving 

ry =, rn =}, r2=[4 =" 
Thus 

rx(0)=3+1=3,  nmx(0)=$-1=-} 


so that, from (1.94), the system response is 


2 
x(t) = yr x(0) ee, = r,x(0) ee, + r,x(0) ee, 


i=1 
That is, 
—t -3t 
x(t) =3e'e,- fee, 


which is in the required spectral form. 


97 


To plot the response, we first draw axes corresponding to the eigenvectors e, and e,, 
as shown in Figure 1.9. Taking these as coordinate axes, we are at the point G ; -}) at 
time t = 0. As ¢ increases, the movement along the direction of e, is much faster than 
that in the direction of e,, since e~’ decreases more rapidly than e“. We can therefore 


guess the trajectory, without plotting, as sketched in Figure 1.9. 
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1.10.6 


We can proceed in an analogous manner to obtain the spectral representation of the 
response to the forced system 


X(t) = Ax(t) + Bu(?) 


with x(t)) given. Making the same assumption regarding the linear independence of the 
eigenvectors e, (i= 1, 2,...,) of the matrix A, the vector Bu(t) may also be written 
as a linear combination of the form 


Bu(t) = 3 Bie; (1.95) 
so that, corresponding to (1.90), we have 
[6,(4) — A,e(t) - B\O)e, +... + [6,0 — A,¢,(0 - B.Ole, = 9 


As a consequence of the linear independence of the eigenvectors e,; this leads to the set 
of uncoupled differential equations 


é(t) — A;c(t) — BAt)=0 (G=1,2,...,7) 


which, using (1.78), have corresponding solutions 


Manse ease | Br) dt (1.96) 


0 


As for c;(t)), the reciprocal basis vectors r; may be used to obtain the coefficients BT). 
Taking the scalar product of both sides of (1.95) with r, and using the relationships 
rie, = 6;, we have 


r,Bu(t)=Bt) (k=1,2,...,n) 
Thus, from (1.96), 


t 
A (t-t9) A(t-9) 
c(t)=e' > 1r,:x(to) | e r,Bu(t) dt 
t 


0 


giving the spectral form of the system response as 


x() => ee, 


i=1 


Canonical representation 

Consider the state-space representation given in (1.69), namely 
x=Ax+ Bu (1.69a) 
y=Cx+Du (1.69b) 


Applying the transformation 
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x=Tz 
where T is a non-singular matrix, leads to 

Tz = ATz + Bu 

y=CTz+ Du 

which may be written in the form 

i=Az+ Bu (1.97a) 

y=Cz+ Du (1.97b) 
where z is now a state vector and 

A=T'AT, B=T'B, C=CT, D=D 


The system input-output relationship is unchanged by the transformation (see Section 
5.7.3), and the linear systems (1.69) and (1.97) are said to be equivalent. By the trans- 
formation the intrinsic properties of the system, such as stability, controllability and 
observability, which are of interest to the engineer, are preserved, and there is merit in 
seeking a transformation leading to a system that is more easily analysed. 

Since the transformation matrix T can be arbitrarily chosen, an infinite number of 
equivalent systems exist. Of particular interest is the case when T is taken to be the 
modal matrix M of the system matrix A; that is, 


T=Me=[e, e ... @, 


where e, (i= 1, 2,..., ) are the eigenvectors of the matrix A. Under the assumption 
that the eigenvalues are distinct, 


A=M'AM-=- A, the spectral matrix of A 
B=M'B 
C=CM D=D 

so that (1.97) becomes 


Z=Az+M'Bu (1.98a) 

y=CMz + Du (1.98b) 
Equation (1.98a) constitutes a system of uncoupled linear differential equations 

Z=A,z,+blu (i=1,2,...,n) (1.99) 
where z = (z;, Z),....,2Z,)' and b) is the ith row of the matrix M™'B. Thus, by reducing 
(1.69) to the equivalent form (1.98) using the transformation x = Mz, the modes 
of the system have been uncoupled, with the new state variables z, (i= 1, 2,..., 7) 


being associated with the ith mode only. The representation (1.98) is called the normal 
or canonical representation of the system equations. 
From (1.78), the solution of (1.99) is 


t 
A (t-to) A(ti-D) 7 . 
z=e X(t) +] e bu(t)dt (i=1,...,n) 
t 


0 
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so that the solution of (1.98a) may be written as 


2(t) =e" a(t) + | eM 'Bu(t)dt (1.100) 


% 


where 


se 0 


A(t-t9) 
e = 


0 nr 
In terms of the original state vector x(t), (1.100) becomes 


M 'x(t)) + | Me“ ?M 'Bu(t)dt (1.101) 


t 


A(t-to) 


x(t) =Mz=Me 


and the system response is then obtained from (1.69b) as 
y(t) = Cx(t) + Duct) 


By comparing the response (1.101) with that in (1.81), we note that the transition matrix 
may be written as 


P(i,%)=6 °° =Me°M” 


The representation (1.98) may be used to readily infer some system properties. If the 
system is stable then each mode must be stable, so, from (1.101), each A, (i= 1, 2,..., 7) 
must have a negative real part. If, for example, the jth row of the matrix M”'B is zero 
then, from (1.99), z,=,z,+ 0, so the input u(t) has no influence on the jth mode of the 
system, and the mode is said to be uncontrollable. A system is said to be controllable 
if all of its modes are controllable. 

If the jth column of the matrix CM is zero then, from (1.98b), the response y is 
independent of z,, so it is not possible to use information about the output to identify z,. 
The state z,is then said to be unobservable, and the overall system is not observable. 


A third-order system is characterized by the state-space model 


0 1 0 1 
x=10 0 1jx+]-3]u, y=[1 0 O]x 
0 -5 -6 18 


where x = [x, x, %3]". Obtain the equivalent canonical representation of the model 
and then obtain the response of the system to a unit step u(t) = H(t) given that initially 
x(0)=[1 1. OJ". 
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The eigenvalues of the matrix 


0 1 0 
A=|0 0 1 
0 -5 -6 


are determined by 


i: 
IA-Al|}=| 0 -A2 1 J=0 
O. -4: 627 


that is, 
MX + 61+ 5)=0 

giving A, = 0, 2, =-1 and 7, =—5, with corresponding eigenvectors 
aol oO Of ~@e=— 4 1% weir = 357 


The corresponding modal and spectral matrices are 


lt a 4 0 0 0 
M=/0 -1 -5|, A=/0 <1 © 
0 1 25 oo Dp 


20 25 4 
M'‘=+| 0 -25 -5 
0 1 


In this case B=[1 -3  18]', so 


20 25 4] 1 20 1 
M'B=3| 0 -25 -5/|-3) =3|-15]=|-i 
0 1 1/|] 18 15 2 


Likewise,C=[1 0 0], giving 


1 1 1 
CM=[! 0 OJ/0 -1 -5;=[1 1 1] 
0 1 25 


Thus, from (1.98), the equivalent canonical state-space representation is 


a, 0 0 Ollz, 1 
a= Z, = -1 0 74) oh ~~] 
Z3 0 0 -5]| z 3 
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Z| 
y=[1 1 1]}z, (1.102b) 


23 


When u(t) = H(t), from (1.100) the solution of (1.102a) is 


e“ 0 0 {1 0 1 
z=|0 e 0 0+ 0 ee"? oO =4|1de 
_ 0 -s(t-7 
0. -%. 0 0 mea ie 
where 
20 24 «#4// 1 “ 
2(0)=M'x(0)=4) 0 -25 -5||1|=|-% 
| 0 ¢ <\/-9 3 
leading to 
1 0 0 ae 1 
z=|0 e' O JI-3 | ee Idr 
0 
0 0 a x 29 
pou t r++ 
=|—fe"|+|—$+2e%| =| -3-Le™ 
xe” a - xe” 2 - te” 


Then, from (1.102b), 


ul -t = 
YH2z+2)+25=(t+5)+(-3-fe°) +(Z- te”) 


_ 8_1 aS —5t 
Htt+; ze 10 © 


If we drop the assumption that the eigenvalues of A are distinct then A= M“"AMis 
no longer diagonal, but may be represented by the corresponding Jordan canonical form 
J with M being made up of both eigenvectors and generalized eigenvectors of A. The 
equivalent canonical form in this case will be 


z=Jz+M'Bu 
y=CMz+ Du 
with the solution corresponding to (1.100) being 


M7 'x(t)) + | Me)? M 'Bu(t)dt 


J (M4) 
x(t)=Me ° 
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66 


1.10.7 Exercises 


Obtain in spectral form the response of the unforced 
second-order system 


(= - 
dato] |= 


4 


‘ 3 3 
X(t) 7? 4 
5 
2 


67 
Using the eigenvectors as the frame of reference, 
sketch the trajectory. 
Using the spectral form of the solution given in 
(1.94), solve the second-order system 
- 2 
x=) 7 le, -x(0)= 
2 -5 3 
and sketch the trajectory. 
68 


Repeat Exercise 62 for the system 


£0) = F “ho. x(0) = 4 
2 -4 2 


Determine the equivalent canonical representation 
of the third-order system 


1 1 -2 -1 
x=|-1 2 ljx+] lju 
0 1 -!l -1 


y=[-2 1 O]x 


The solution of a third-order linear system is 
given by 


X= Oe 'e, + Ae “e, + A,e “e, 
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where @p, e, and e, are linearly independent vectors 
having values 


e=[1 1 of, 
e=[1 2 3] 


e,=[0 1 17, 


Initially, at time ¢ = 0 the system state is 
x(0) = [1 1. 1)". Find a, a, and @, using 
the reciprocal basis method. 


Obtain the eigenvalues and eigenvectors of the matrix 


A= 5 4 
1 2 
Using a suitable transformation x(t) = M(t), reduce 
X(t) = Ax(f) to the canonical form Z(t) = Az(A), 
where A is the spectral matrix of A. Solve the 


decoupled canonical form for z, and hence solve 
for x(f) given that x(0)=[1  4]’. 


A second-order system is governed by the state 
equation 


«|; to]? eo (¢ = 0) 
21 1 


Using a suitable transformation x(t) = Mz(t), reduce 
this to the canonical form 


2(t) = Az(t) + Bu(t) 


where A is the spectral matrix of 


E 


and B is a suitable 2 x 2 matrix. 

For the input u(#)=[4 3]" solve the decoupled 
canonical form for z, and hence solve for x(t) given 
that x(0) =[1 2]'. Compare the answer with that 
for Exercise 60. 


In Chapter 5 we shall consider the solution of state-space models using the Laplace 
transform method and in Chapter 6 extend the analysis to discrete-time systems using 


z-transforms. 
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Aa ee a leddacxvalarcar-)ye)ie-ieteaey Lyapunov stability analysis 


The Russian mathematician Alexsander Mikhailovich Lyapunov (1876-1918) devel- 
oped an approach to stability analysis which is now referred to as the direct (or second) 
method of Lyapunov. His approach remained almost unknown in the English-speaking 
world for around half a century, before it was translated into English in the late 1950s. 
Publication of Lyapunov’s work in English aroused great interest, and it is now widely 
used for stability analysis of linear and non-linear systems, both time-invariant and 
time-varying. Also, the approach has proved to be a useful tool in system design such 
as, for example, in the design of stable adaptive control systems. The Lyapunov method 
is in fact a ‘method of approach’ rather than a systematic means of investigating stability 
and much depends on the ingenuity of the user in obtaining suitable Lyapunov func- 
tions. There is no unique Lyapunov function for a given system. 

In this section we briefly introduce the Lyapunov approach and will restrict con- 
sideration to the unforced (absence of any input) linear time-invariant system 


x= Ax (1.103) 


where x = [x,, X),...,X,]' is the n-state vector and A is a constant n x n matrix. For the 
linear system (1.103) the origin x = 0 is the only point of equilibrium. If, for any initial state 
x(0), the trajectory (solution path) x(4) of the system approaches zero (the equilibrium point) 
as tf — co then the system is said to be asymptotically stable. In practice the elements 
of the matrix A may include system parameters and we are interested in determining 
what constraints, if any, must be placed on these parameters to ensure system stability. 
Stability of (1.103) is further discussed in Section (5.7.1), where algebraic criteria for 
stability are presented. In particular, it is shown that stability of system (1.103) is 
ensured if and only if all the eigenvalues of the state matrix A have negative real parts. 

To develop the Lyapunov approach we set up a nest of closed surfaces, around the 
origin (equilibrium point), defined by the scalar function 


V(x) = VixXy, Xy, -- XJ =C (1.104) 


where C is a positive constant (the various surfaces are obtained by increasing the 
values of C as we move away from the origin). If the function V(x) satisfies the follow- 
ing conditions: 


(a) V(x) = 0 at the origin, that is V(0) = 0; 
(b) V(x) > 0 away from the origin; 
(c) V(x) is continuous with continuous partial derivatives; 


then it is called a scalar Lyapunov function. (Note that conditions (a) and (b) together 
ensure that V(x) is a positive definite function.) We now consider the rate of change 
of V(x), called the Eulerian derivative of V(x) and denoted by V(x), along the trajectory 
of the system under investigation; that is, 

— AV Ax dV dx, 2 ae 49V Ax, 


Vix) =—-— + 
i Ox, dt ax, dt ox, At 


(1.105) 


where the values of x), X,,..., x, are substituted from the given equations representing 
the system ((1.103) in the case of the linear equations under consideration). 

If V satisfies the condition 

(d) V(x) is negative definite 


then it follows that all the trajectories cross the surfaces V(x) = C in an inward direction 
and must tend to the origin, the position of equilibrium. Thus asymptotic stability has 
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been assured without having to solve the differential equations representing the system. 
The function V(x) which satisfies conditions (a)—(d) is called a Lyapunov function for 
the system being considered. 

If we start with a positive-definite V(x) and impose conditions on V(x) to be negative- 
definite, then these conditions will provide sufficient but not necessary stability criteria, 
and in many cases they may be unduly restrictive. However, if we are able to start with 
a negative-definite V(x) and work back to impose conditions on V(x) to be positive- 
definite, then these conditions provide necessary and sufficient stability criteria. 
This second procedure is far more difficult to apply than the first, although it may be 
applied in certain cases, and in particular to linear systems. 

Of particular importance as Lyapunov functions for linear systems are quadratic 
forms in the variables x,, x5,..., x, which were introduced in Section 1.6.4. These 
may be written in the matrix form V(x) = x'Px, where P is a real symmetric matrix. 
Necessary and sufficient conditions for V(x) to be positive-definite are provided by 
Sylvester’s criterion, which states that all the principal minors of P of order 1, 2,...,7 
must be positive; that is 


Pu Pi Pr 
Pp Pp 
Pu > 9, . i >0,|Pi2 Pr» Pr oO, 225 (P| 20 
Pi2 P2 
Pw P23 P33 


Returning to the linear system (1.103) let us consider as a tentative Lyapunov function 
the quadratic form 


V(x) = x'Px 


where P is an n Xn real symmetric matrix. To obtain the Eulerian derivative of V(x) 
with respect to system (1.103) we first differentiate V(x) with respect to ¢ 


a _ Px + x"Pe 
dt 


and then substitute for x’ and x from (1.103) giving 
V(x) = (Ax)'Px + x™P(Ax) 
V(x) =x'(A'P + PA)x 
or alternatively 
V(x) = -x'Ox (1.106) 
-Q=A'P+PA (1.107) 


that is 


where 


To obtain necessary and sufficient conditions for the stability of the linear system 
(1.103) we start with any negative definite quadratic form —x'Qx, with an n x n 
symmetric matrix Q, and solve matrix equation (1.107) for the elements of P. The con- 
ditions imposed on P to ensure that it is positive definite then provide the required 
necessary and sufficient stability criteria. 


The vector-matrix differential equation model representing an unforced linear R-C 
circuit is 
x= -4a 4a x (i) 
2a -6a 


Examine its stability using the Lyapunov approach. 
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Take Q of equation (1.107) to be the identity matrix I which is positive-definite 


(thus —Q is negative-definite), then (1.107) may be written 


-1 0 _|-4a 20) |Piu 
0 - 4a -6a Pi2 


Equating elements in (ii) gives 


2 4 Pu Pi2||-4a 4a (ii) 
P2 Pr Pr|| 20 -6a 


—8ap,,+4ap,,=—1, 4ap,, — 10ap,, + 2ap,, = 0, 8ap,, — 12ap» =-1 


Solving for the elements gives 


1 


Se 
Pa 4007 P i0@ 


so that 


Pek! = 
40a!14 6 


The principal minors of ; j are |7| > 0 and ‘i 


a =26> 0. 
6 


Thus, by Sylvester’s criterion, P is positive-definite and the system is asymptotically 


stable provided a > 0. 


Note that the Lyapunov function in this case was 


1 
V(x) =x'Px = tia (7x} + 8x,x, + 6x3) 


1.11.1 Exercises 


Using the Lyapunov approach investigate the 
stability of the system described by the state 
equation 


#2 fe ic 72 
3 = 


Take Q to be the unit matrix. Confirm your answer 
by determining the eigenvalues of the state matrix. 


Repeat Exercise 68 for the system described by the 
state equation 


a 
x= x 
=f i] 


For the system modelled by the state equation 


“AE 


use the Lyapunov approach to determine the 
constraints on the parameters a and b that yield 
necessary and sufficient conditions for asymptotic 
stability. 


Condition (d) in the formulation of a Lyapunov 
function, requiring V(x) to be positive-definite, may 
be relaxed to V(x) being positive-semidefinite 
provided V(x) is not identically zero along any 
trajectory. A third-order system, in the absence of 
an input, is modelled by the state equation 


x= Ax 


where x =[x; %) %,]' and 


0 1 0 

1 | with & being a constant scalar. 
-k 0 -l 

It is required to use the Lyapunov approach to 

determine the constraints on k to ensure asymptotic 

stability. 
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(a) In (1.106) choose Q to be the positive- 
semidefinite matrix 


so that 
V(x) = -x'Qx = —x} 


Verify that V(x) is identically zero only at the 
origin (equilibrium point) and is therefore not 
identically zero along any trajectory. 


(b) Using this matrix Q solve the matrix equation 
A'P+PA=-Q 


to determine the matrix P. 


1.12 


(c) Using Sylvester’s criterion show that the 
system is asymptotically stable for 
0<k<6. 


A feedback control system modelled by the 
differential equation 


X¥+axt+kx=0 


is known to be asymptotically stable, for k > 0, 
a > 0. Set up the state-space form of the equation 
and show that 


V(x1, 2) = hog + (> + ax), x) =X, xy = 


is a suitable Lyapunov function for verifying 
this. 


aedalalircarlyieeideee Capacitor microphone 


Many smaller portable tape recorders have a capacitor microphone built in, since such 
a system is simple and robust. It works on the principle that if the distance between the 
plates of a capacitor changes then the capacitance changes in a known manner, and 
these changes induce a current in an electric circuit. This current can then be amplified 
or stored. The basic system is illustrated in Figure 1.10. There is a small air gap (about 
0.02 mm) between the moving diaphragm and the fixed plate. Sound waves falling on 
the diaphragm cause vibrations and small variations in the capacitance C; these are 


certainly sufficiently small that the equations can be linearized. 


Figure 1.10 Capacitor Air gap Moving diaphragm 
microphone. 
Fixed plate 
Insulation 
Metal frame 


We assume that the diaphragm has mass m and moves as a single unit so that its 
motion is one-dimensional. The housing of the diaphragm is modelled as a spring- 
and-dashpot system. The plates are connected through a simple circuit containing a 
resistance and an imposed steady voltage from a battery. Figure 1.11 illustrates the 
model. The distance x(t) is measured from the position of zero spring tension, F is the 
imposed force and fis the force required to hold the moving plate in position against 
the electrical attraction. The mechanical motion is governed by Newton’s equation 
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Figure 1.11 Capacitor 


; ‘—» x(t) from zero-spring-tension 
microphone model. 


= dq/dt position 
> ; N 
f 
a 
k 
|| 
R Fixed Moving FO) 


plate diaphragm 


mi =—kx -—Ax—f+ F (1.108) 


and the electrical circuit equation gives 
ee a ee (1.109) 
Cc dt 


The variation of capacitance C with x is given by the standard formula 


_ Coa 


atx 


where a is the equilibrium distance between the plates. The force fis not so obvious, 
but the following assumption is standard 


2 
2d ( 1 ) q 
=lg°—|-|=14 
f-24 dx\C/ *Cya 
It is convenient to write the equations in the first-order form 


X=v 
2 
14 
= + F(t 
Ca (t) 
+ 
i= 
a Co 
Furthermore, it is convenient to non-dimensionalize the equations. While it is obvious 
how to do this for the distance and velocity, for the time and the charge it is less so. 
There are three natural time scales in the problem: the electrical time tT; = RC), the 
spring time T; = m/k and the damping time T, = m/A. Choosing to non-dimensionalize 
the time with respect to T,, the non-dimensionalization of the charge follows: 


mv =—kx — Av - 


tT=-+, x=%, y-—, 0=—2. 
Ty a kala (2Cyka’) 


Then, denoting differentiation with respect to t by a prime, the equations are 


; Cok 
XxX’ = —V 
A 

m , 2 F 

ft =F Pa ee 

ARC, Ott 
Q’ =-O(1 + X)+ 2S 
\(2C ka’) 
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Figure 1.12 Solutions 
to equations (1.111). 
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There are four non-dimensional parameters: the external force divided by the spring 
force gives the first, G = F/ka; the electrical force divided by the spring force gives the 
second, D* = (E’C,/2a)/ka; and the remaining two are 


A= RCok _ 1173 ee ee 
A nm ARC, i; 
The final equations are therefore 
X’=AV 
BV’=-X-V-Q°+G (1.110) 


O’=-Q(1+X)+D 
In equilibrium, with no driving force, G = 0 and V=X’ = V’ = Q’ =0, so that 
2 — 
g ee, (1.111) 
O(1+X)- D=0 
or, on eliminating Q, 
X(1+XYy =-D’ 
From Figure 1.12, we see that there is always one solution for ¥ < —1, or equivalently 
x <-—a. The implication of this solution is that the plates have crossed. This is clearly 
impossible, so the solution is discarded on physical grounds. There are two other solu- 
tions if 
2 
p< i(o'=$ 
or 


2 
es (1.112) 
2ka 


27 


We can interpret this statement as saying that the electrical force must not be too strong, 
and (1.112) gives a precise meaning to what ‘too strong’ means. There are two 
physically satisfactory equilibrium solutions —} <_X, < 0 and —1 <_X, < -}, and the 
only question left is whether they are stable or unstable. 

Stability is determined by small oscillations about the two values X, and_X,, where 
these values satisfy (1.111). Writing 


X=X,+ €, O=0,+N, V=0 
and substituting into (1.110), neglecting terms in €’, y*, 6°, €@ and so on, gives 
€’ =A 
BQ’ =-e- 6-20.n (1.113) 
n’=(-Qe-(1+X;)n) 


Equations (1.113) are the linearized versions of (1.110) about the equilibrium values. 
To test for stability, we put G= 0 and e= Le’, 0= Me™, n= Ne" into (1.113): 


La =AM 
BMa =-L —M-20,N 
Na =-O,L-(1+X)N 
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which can be written in the matrix form 


£ 0 A 0 re 
a|M|=|-1/B -1/B -20,/B||M 
N = © —=(142)/|" 


Thus the fundamental stability problem is an eigenvalue problem, a result common 
to all vibrational stability problems. The equations have non-trivial solutions if 


-a A 0 
0O= |-1/B -(1/B)-a —20,/B 
-2, 0 -(1+X;) — oF 


=—-[Boe + (BU +X) + lho + (14+ X,4+ Ala + A(1 + X,- 207)/B 


For stability, @ must have a negative real part, so that the vibrations damp out, and the 
Routh—Hurwitz criterion (Section 5.6.2) gives the conditions for this to be the case. 
Each of the coefficients must be positive, and for the first three 


B>0O, BA+X)+1> 0, 1+X+A>0 

are obviously satisfied since —1 <_X,< 0. The next condition is 
A(1 +.X,;- 207) >0 

which, from (6.118), gives 
14+ 3X;> 0, or X,>-} 


Thus the only solution that can possibly be stable is the one for which X; > —}; the other 
solution is unstable. There is one final condition to check, 


[B(1 +X) + 1]. +.X,+ 4) - BA(1 +. X,- 207) > 0 
or 
BU +X)+14+X,+4+2BAQ; > 0 


Since all the terms are positive, the solution X, > ; is indeed a stable solution. 

Having established the stability of one of the positions of the capacitor diaphragm, 
the next step is to look at the response of the microphone to various inputs. The char- 
acteristics can most easily be checked by looking at the frequency response, which is 
the system response to an individual input G = be’, as the frequency @ varies. This 
will give information of how the electrical output behaves and for which range of 
frequencies the response is reasonably flat. 

The essential point of this example is to show that a practical vibrational problem 
gives a stability problem that involves eigenvalues and a response that involves a 
matrix inversion. The same behaviour is observed for more complicated vibrational 
problems. 
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1.13 Review exercises (1-20) 


Check your answers using MATLAB or MAPLE whenever possible. 


i 


2 


3 


4 


Obtain the eigenvalues and corresponding 
eigenvectors of the matrices 


=I 6 12 
(a) | 0 -13 30 
| 0 -9 20 
| oe i 
(Oy) eet 
I=i 2 
Pit - © 
Gr let 2 21 
Oo =1 i 


Find the principal stress values (eigenvalues) 
and the corresponding principal stress directions 
(eigenvectors) for the stress matrix 


3) 

= ||2 

1 

Verify that the principal stress directions are 


mutually orthogonal. 


Find the values of b and c for which the matrix 


2 -1 0 
A=/-1 3 
0 by @ 


has[1 0 1]' as an eigenvector. For these 
values of 6 and c calculate all the eigenvalues 


and corresponding eigenvectors of the matrix A. 


Use Gerschgorin’s theorem to show that the 
largest-modulus eigenvalue 1, of the matrix 


At 
A=|-1 4 -1 
Ot 4 


is such that 2 <|A,| <6. 

Use the power method, with starting vector 
x%=[-1 1 -1]',to find A, correct to one 
decimal place. 
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(a) Using the power method find the dominant 
eigenvalue and the corresponding eigenvector 
of the matrix 


ell 1 
A=)1 25 1 
il 3 


starting with an initial vector[1 1 1]' 
and working to three decimal places. 


(b) Given that another eigenvalue of A is 1.19 
correct to two decimal places, find the value of the 
third eigenvalue using a property of matrices. 

(c) Having determined all the eigenvalues of A, 
indicate which of these can be obtained by 
using the power method on the following 
matrices: (i) A™'; (ii) A — 31. 


Consider the differential equations 


dx 
= =4xy+ yt 
a Kastan actaees 


SY = 2x + 5y +42 


dz _ 

dt 
Show that if it is assumed that there are solutions 
of the form x = ae“, y = Be” and z= ye“ then 
the system of equations can be transformed into 
the eigenvalue problem 


xy 


4 Ie IL Ife a 
2 5 4|/B =A\B 
= =i @ll||% Y 


Show that the eigenvalues for this problem 
are 5, 3 and 1, and find the eigenvectors 
corresponding to the smallest eigenvalue. 


Find the eigenvalues and corresponding 
eigenvectors for the matrix 


CP eh ad 
pele ee oy) 
ell 


Write down the modal matrix Mand spectral 
matrix A of A, and confirm that 


M'AM=A 
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10 


Show that the eigenvalues of the symmetric matrix 


4 
A=| 0 4 
=p 3 


are 9, 3 and —3. Obtain the corresponding 
eigenvectors in normalized form, and write down 
the normalized modal matrix M. Confirm that 


MAM=A 


where A is the spectral matrix of A. 


In a radioactive series consisting of four different 
nuclides starting with the parent substance N, and 
ending with the stable product N, the amounts of 
each nuclide present at time ¢ are given by the 
differential equations model 


oH oy 
oe = 6N,-4N, 
os = 4N, - 2N, 
aN ay, 


Express these in the vector—matrix form 

N=AN 
where N=[N, WN, N, N,]'. Find the eigenvalues 
and corresponding eigenvectors of A. Using the 


spectral form of the solution, determine N,(t) given 
that at time t= 0, N,= C and N, = N,=N,=0. 


(a) Given 


2 
A= 
Wil 
use the Cayley—Hamilton theorem to find 


(i) A’—3A° + A*+ 3A} - 2A? +31 
(ii) A*, where k > 0 is an integer. 


(b) Using the Cayley—Hamilton theorem, find 
e4’ when 


“{. 


iL 


12 


is 


Show that the matrix 


IL 2 3 
A=/0 1 4 
O 


has an eigenvalue A = 1 with algebraic 
multiplicity 3. By considering the rank of a 
suitable matrix, show that there is only one 
corresponding linearly independent eigenvector 
e,. Obtain the eigenvector e, and two further 
generalized eigenvectors. Write down the 
corresponding modal matrix M and confirm that 
M'AMBE J, where J is the appropriate Jordan 
matrix. (Hint: In this example care must be taken 
in applying the procedure to evaluate the 
generalized eigenvectors to ensure that the 
triad of vectors takes the form {T’a@, Ta, @}, 
where T=A — AL with T’w=e,.) 


The equations of motion of three equal masses 
connected by springs of equal stiffness are 


X¥=-2x+y 
Pex-2yt+z 
Z=y—2z 


Show that for normal modes of oscillation 
Xi —PXICOSOE: y= Ycosat, 
z=Zcosat 


to exist then the condition on A= @ is 


A-2 

Find the three values of A that satisfy this 
condition, and find the ratios X: Y: Zin 
each case. 

Classify the following quadratic forms: 

(a) 2x° +3? + 22° — 2xy — 2yz 

(b) 3x° + 7y? + 227 — 4xy — 4xz 

(c) 16x? + 36)? + 172 + 32xy + 32xz +1 6yz 
(d) —21x* + 30xy — 12xz 


(e) x’ - 37° 


11? + 8yz — 22° 
52° + Ixy + 2xz + 2yz 
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14 


15 


16 


Show thate,=[1 2 
the matrix 


3]' is an eigenvector of 


ll 
Bi vin 
| 

NIw ee Nie 


Niw 


and find its corresponding eigenvalue. Find the 
other two eigenvalues and their corresponding 
eigenvectors. 

Write down in spectral form the general 
solution of the system of differential 
equations 

dx 
B= = eas 
dt i 
dy 
= 4 _ 
Go 


282 =—3x43y +2 


Hence show that if x = 2, y=4 andz=6 
when ¢ = 0 then the solution is 


t 


x =2e', y=4e', OCs 


(a) Find the SVD form of the matrix 
12 09 -4 
A = 
WG 12 3 
(b) Use the SVD to determine the pseudo inverse 


A‘ and confirm it is a right inverse of A. 


(c) Determine the pseudo inverse At without using 
the SVD. 


From (1.51) the unitary matrices U and V and sigma 
matrix 2 may be written in the partitioned form: 


where S is r x r diagonal matrix having the singular 
values of A as its diagonal elements and 0 denotes 
zero matrices having appropriate order. 


(a) Show that the SVD form of A may be 
expressed in the form 


AS UsSUe 
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This is called the reduced singular value 
decomposition of A. 


(b) Deduce that the pseudo inverse is given by 
A=PS'O" 


(c) Use the results of (a) and (b) to determine 
the SVD form and pseudo inverse of the 


matrix 
| es 
A=/|-2 2 
2 -2 


and check your answers with those obtained 
in Exercise 46. 


A linear time-invariant system (A, b, c) is 
modelled by the state-space equations 


X(t) = Ax(t) + bu(t) 
y(t) = c'x(t) 


where x(t) is the n-dimensional state vector, and 
u(t) and y(¢) are the system input and output 
respectively. Given that the system matrix A 
has n distinct non-zero eigenvalues, show that 
the system equations may be reduced to the 
canonical form 


E() = AE(t) + bul) 

yD = e1§) 
where A is a diagonal matrix. What properties of 
this canonical form determine the controllability 
and observability of (A, b, c)? 


Reduce to canonical form the system (A, b, c) 
having 


owt 
Nell 9 1 

| 0 1 -1 

| -1 =) 
b= 1 c= il 

|-1 0 


and comment on its stability, controllability and 
observability by considering the ranks of the 
appropriate Kalman matrices [b Ab A/’b] 
and[e At‘te (A‘)c]. 
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18 


19 


20 


A third-order system is modelled by the state-space 
representation 


wherex=[x, x, x,]/'andu=[u, u,]". 
Find the transformation x = Mz which reduces 
the model to canonical form and solve for x(¢) 
given x(0)=[10 5 2]'’ andu(f)=[t 1]'. 


The behaviour of an unforced mechanical system 
is governed by the differential equation 


5 2 -l 0 
R(t)=|3 6 —9)x(t), x(0)=/1 
ie aoe 0 


(a) Show that the eigenvalues of the system 
matrix are 6, 3, 3 and that there is only 
one linearly independent eigenvector 
corresponding to the eigenvalue 3. Obtain the 
eigenvectors corresponding to the eigenvalues 
6 and 3 and a further generalized eigenvector 
for the eigenvalue 3. 


(b) Write down a generalized modal matrix M 
and confirm that 


AM=M) 
for an appropriate Jordan matrix J . 
(c) Using the result 
x(t) = Me!’M™x(0) 
obtain the solution to the given differential 


equation. 


(Extended problem) Many vibrational systems are 
modelled by the vector—matrix differential equation 


X(t) = Ax(1) () 


where A is a constant n x n matrix and 
x) =[%()  x,(4) x,(O]". By 
substituting x = eu, show that 


Ru= Au (2) 


and that non-trivial solutions for u exist 
provided that 


|A — #1 =0 (3) 
Let Aj, 43,..., A; be the solutions of (3) and 
U,, Uy, ..., u, the corresponding solutions of (2). 


Define M to be the matrix having u,, u,..., 
u,, as its columns and S to be the diagonal matrix 
having 44, 23, ..., A? as its diagonal elements. 
By applying the transformation x(t) = Mq(¢), 
where q(t) =[q(t) q(t) qn(t)]', to (1), 
show that 


G=Sq (4) 
and deduce that (4) has solutions of the form 
q; = C, sin(@,t + a,) (5) 


where c; and a; are arbitrary constants and 
A; =j@,, with j = \(-1). 

The solutions A? of (3) define the natural 
frequencies @, of the system. The corresponding 
solutions g; given in (5) are called the normal 
modes of the system. The general solution of (1) 
is then obtained using x(t) = Mg(¢). 

A mass-spring vibrating system is governed 
by the differential equations 


¥,(t) = -3x,(t) + 2x,(t) 
¥,(t) = x(t) — 2x2(t) 


with x,(0) = 1 and x,(0) = x,(0) = x,(0) = 2. 
Determine the natural frequencies and the 
corresponding normal modes of the system. 
Hence obtain the general displacement x,(t) 
and x,(t) at time ¢ > 0. Plot graphs of both 
the normal modes and the general solutions. 
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Yas Introduction 


Frequently the equations which express mathematical models in both engineering ana- 
lysis and engineering design involve derivatives and integrals of the models’ variables. 
Equations involving derivatives are called differential equations and those which include 
integrals or both integrals and derivatives are called integral equations or integro- 
differential equations. Generally integral and integro-differential equations are more 
difficult to deal with than purely differential ones. 

There are many methods and techniques for the analytical solution of elementary 
ordinary differential equations. The most common of these are covered in most first- 
level books on engineering mathematics (e.g. Modern Engineering Mathematics). 
However, many differential equations of interest to engineers are not amenable to ana- 
lytical solution and in these cases we must resort to numerical solutions. Numerical 
solutions have many disadvantages (it is, for instance, much less obvious how changes 
of parameters or coefficients in the equations affect the solutions) so an analytical solu- 
tion is generally more useful where one is available. 

There are many tools available to the engineer which will provide numerical solutions 
to differential equations. The most versatile of these perhaps are the major computer 
algebra systems such as MAPLE. These contain functions for both analytical and 

numerical solution of differential equations. Systems such as MATLAB/Simulink and 
Mathcad can also provide numerical solutions to differential equations problems. It 
may sometimes be necessary for the engineer to write a computer program to solve 
a differential equation numerically, either because suitable software packages are 
not available or because the packages available provide no method suitable for the 
particular differential equation under consideration. 

Whether the engineer uses a software package or writes a computer program for 
the specific problem, it is necessary to understand something of how numerical 
solution of differential equations is achieved mathematically. The engineer who 
does not have this understanding cannot critically evaluate the results provided by a 
software package and may fall into the trap of inadvertently using invalid results. In 
this chapter we develop the basics of the numerical solution of ordinary differential 
equations. 


2.2 Engineering application: Hitieimiiee Musee ire 


The problem of determining the motion of a body falling through a viscous fluid arises 
in a wide variety of engineering contexts. One obvious example is that of a parachutist, 
both in free fall and after opening his or her parachute. The dropping of supplies from 
aircraft provides another example. Many industrial processes involve adding particulate 
raw materials into process vessels containing fluids, whether gases or liquids, which 
exert viscous forces on the particles. Often the motion of the raw materials in the pro- 
cess vessel must be understood in order to ensure that the process is effective and 
efficient. Fluidized bed combustion furnaces involve effectively suspending particles 
in a moving gas stream through the viscous forces exerted by the gas on the particles. 
Thus, understanding the mechanics of the motion of a particle through a viscous fluid 
has important engineering applications. 
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dS 


mg 


Figure 2.1 A particle 
falling through a 
viscous fluid. 


When a particle is falling through a viscous fluid it may be modelled simply in the 
following way. The force of gravity acts downwards and is opposed by a viscous drag 
force produced by the resistance of the fluid. Figure 2.1 shows a free body diagram of 
the particle which is assumed to be falling vertically downwards. If the particle’s mass 
is m, the gravitational force is mg, and it is opposed by a drag force, D, acting to oppose 
motion. The displacement of the particle from its initial position is x. 

The equation of motion is 


dx _ 
m——=mg—D (2.1) 
dt 
Before we can solve this equation, the form of the drag term must be determined. 
For particles moving at a high speed it is often assumed that the drag is proportional to 
the square of the speed. For slow motion the drag is sometimes assumed to be directly 


proportional to the speed. In other applications it is more appropriate to assume that 
drag is proportional to some power of the velocity, so that 


D=k*= (&) where, normally, | << a@< 2 


The differential equation (2.1) then becomes 


dx _ (2) 
m— = —k = 
dé dt 
2 a 
Le. ne + (S) =mg (2.2) 


This is a second-order, nonlinear, ordinary differential equation for x, the displacement of 
the particle, as a function of time. In fact, for both w= 1 and @ = 2, (2.2) can be solved 
analytically, but for other values of a@ no such solution exists. If we want to solve the 
differential equation for such values of @ we must resort to numerical techniques. 


Numerical solution of first-order ordinary 
differential equations 


In a book such as this we cannot hope to cover all of the many numerical techniques which 
have been developed for dealing with ordinary differential equations so we will concen- 
trate on presenting a selection of methods which illustrate the main strands of the theory. 
In so doing we will meet the main theoretical tools and unifying concepts of the area. 
In the last twenty years great advances have been made in the application of computers 
to the solution of differential equations, particularly using computer algebra packages 
to assist in the derivation of analytical solutions and the computation of numerical solu- 
tions. The MATLAB package is principally oriented towards the solution of numerical 
problems (although its Symbolic Math Toolbox and the MuPAD version are highly 
capable) and contains a comprehensive selection of the best modern numerical techniques 
giving the ability to solve most numerical problems in ordinary differential equations. 
Indeed numerical solutions can be achieved both in native MATLAB and in the Simulink 
simulation sub-system; which of these paths the user chooses to follow may well be 
dictated as much by their experience and professional orientation as by theoretical 
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2.3.1 


Figure 2.2 

The direction field 
for the equation 
dx/dt = x(1 — x)t. 


considerations. MAPLE, despite being mainly orientated towards the solution of sym- 
bolic problems, also contains a comprehensive suite of numerical solution routines and 
is, in practice, just as capable as MATLAB in this area. Moreover, MAPLE gives to the 
user more control of the solution method used and includes a number of ‘classical’ 
solution methods. These classical methods include all the methods which are used, in 
this chapter, to introduce, develop and analyse the main strands of the theory mentioned 
above. For this reason, MAPLE will be featured rather more frequently than MATLAB, 
but the practising engineer is as likely to be using MATLAB for the numerical solution 
of real-world problems as using MAPLE. 

Despite the fact that professional engineers are very likely to be using these packages 
to compute numerical solutions of ordinary differential equations it is still important 
that they understand the methods which the computer packages use to do their work, for 
otherwise they are at the mercy of the decisions made by the designers of the packages 
who have no foreknowledge of the applications to which users may put the package. If 
the engineering user does not have a sound understanding of the principles being used 
within the package there is the ever present danger of using results outside their domain 
of validity. From there it is a short step to engineering failures and human disasters. 


A simple solution method: Euler’s method 


For a first-order differential equation dx/dt = f(t, x) we can define a direction field. The 
direction field is that two-dimensional vector field in which the vector at any point (¢, x) 
has the gradient dx/dt. More precisely, it is the field 


LL) 11, f(t, 0) 

vE1 + f(t, x) J 
For instance, Figure 2.2 shows the direction field of the differential equation dx/dt = 
x(1 — x)t. 

Since a solution of a differential equation is a function x(t) which has the property 
dx/dt = f(t, x) at all points (¢, x) the solutions of the differential equation are curves in 
the (t, x) plane to which the direction field lines are tangential at every point. For 
instance, the curves shown in Figure 2.3 are solutions of the differential equation 
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Figure 2.3 Solutions 
of dx/dt = x(1 — x)t 
superimposed on its 
direction field. 


Figure 2.4 

The construction of 
a numerical solution 
of the equation 
dx/dt = f(t, x). 


& =a eset 


This immediately suggests that a curve representing a solution can be obtained by 
sketching on the direction field a curve that is always tangential to the lines of the 
direction field. In Figure 2.4 a way of systematically constructing an approximation to 
such a curve is shown. 

Starting at some point (¢), x9), a straight line parallel to the direction field at that 
point, f(t, Xo), is drawn. This line is followed to a point with abscissa ft) + h. The ordin- 
ate at this point is x) + Af(t, xX»), which we shall call _X,. The value of the direction field 
at this new point is calculated, and another straight line from this point with the new 
gradient is drawn. This line is followed as far as the point with abscissa tf, + 2h. The 
process can be repeated any number of times, and a curve in the (¢, x) plane consisting 
of a number of short straight-line segments is constructed. The curve is completely 
defined by the points at which the line segments join, and these can obviously be 
described by the equations 
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Figure 2.5 The 


: x Analytic 
Euler-method solutions Pe hha 
ey ae 
of dx/dt = x*te” for ~_-h = 0.0125 


h = 0.05, 0.025 and 10 
0.0125. ~h = 0.025 


Sep 0.05 


a le 


h=h+h, X, =X + Af(to, Xo) 
h=t th, X, =X, + hf(t, X) 
h=t,+h, X3 =X) + hf(h, X) 


boat = t, oe h, Xie = X, + hf(t,, X,) 


These define, mathematically, the simplest method for integrating first-order differential 
equations. It is called Euler’s method. Solutions are constructed step by step, starting 
from some given starting point (fp, x)). For a given f, each different x, will give rise to 
a different solution curve. These curves are all solutions of the differential equation, but 


each corresponds to a different initial condition. 


The solution curves constructed using this method are obviously not exact solutions 
but only approximations to solutions, because they are only tangential to the direction 
field at certain points. Between these points, the curves are only approximately tangen- 
tial to the direction field. Intuitively, we expect that, as the distance for which we follow 
each straight-line segment is reduced, the curve we are constructing will become a 
better and better approximation to the exact solution. The increment / in the independent 
variable ¢ along each straight-line segment is called the step size used in the solution. 


In Figure 2.5 three approximate solutions of the initial-value problem 


=xte", x(0)=0.91 


for step sizes h = 0.05, 0.025 and 0.0125 are shown. These steps are sufficiently small 
that the curves, despite being composed of a series of short straight lines, give the illusion 
of being smooth curves. The equation (2.3) actually has an analytical solution, which 


can be obtained by separation: 


1 


x = 
(l+fe'+C 


The analytical solution to the initial-value problem is also shown in Figure 2.5 for com- 
parison. It can be seen that, as we expect intuitively, the smaller the step size the more 


closely the numerical solution approximates the analytical solution. 
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2) MAPLE provides options in the dsolve function, the general-purpose ordinary 
differential equation solver, to return a numerical solution computed using the Euler 
method. Using this option we can easily generate the solutions plotted on Figure 2.5. 
In fact we can readily extend the figure to some smaller time steps. The following 
MAPLE worksheet will produce a figure similar to Figure 2.5 comparing the solu- 
tions obtained from the Euler method using time steps of 0.05, 0.025, 0.0125, 
0.00625, 0.003125 and the exact solution. The pattern established in Figure 2.5 can be 
seen to continue with each halving of the time step producing a solution with a yet 
smaller error when compared with the exact solution. 

> deql:=diff (x(t) ,t)=x(t)*2*t*exp(-t);initl:=x(0)=0.91; 
> #solve the differential equation with 5 different 


timesteps 
> x<ileaclsolya(iclecgil, aliatieil} , 

numeric,method=classical [foreuler] , output=listprocedure, 
stepsize=0.05); 
= x2oeclsolivva(iceeil, alianicil} , 
numeric,method=classical [foreuler] , output=listprocedure, 
stepsize=0.025); 
> <iececlsolya(iclegil, alia}, 
numeric,method=classical [foreuler] , output=listprocedure, 
Sie Wea), OLAS) 
= lcaclsoliwyvea(iclecgil, aliaticil}, 
numeric,method=classical [foreuler] , output=listprocedure, 
stepsize=0.00625); 


=> -Seaceolwa(iceel, aime}, 


numeric,method=classical [foreuler] , output=listprocedure, 
stepsize=0.003125 
> #extract the five solutions from the listprocedure 


structures 
roe i itcom Il to 5 corsolwiciom | reso (2,s<| | 12) peiac clog 
#find the exact solution 
xa: =—dsolve({deql, ainatl})); 
#plot the five numerical solutions and the exact solution 
lloe ( (Sec (solwciwem| |a Ce), teal. .5) ,cat(2,#a) (ec) ] ~c=0. 12) 5 


Wi WA ONE SWE OY 


Example 2.1 The function x() satisfies the differential equation 


dx _x+t 
dt xt 


and the initial condition x(1) = 2. Use Euler’s method to obtain an approximation to the 
value of x(2) using a step size of h = 0.1. 


Solution In this example the initial value of tis 1 and x(1) = 2. Using the notation above we have 
ty = 1, and x) = 2. The function f(t, x) = So we have 


t,=%+h=1+0.1 = 1.1000 
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Figure 2.6 
Computational results 
for Example 2.1. 


EY 


2.3.2 


t xX X+t Xt pest 

Xt 
1,0000 2.0000 3.0000 2.0000 0.1500 
1.1000 2.1500 3.2500 2.3650 0.1374 
1.2000 2.2874 3.4874 2.7449 0.1271 
1,3000 2.4145 3.7145 3.1388 0.1183 
1.4000 2.5328 3.9328 3.5459 0.1109 
1.5000 2.6437 4.1437 3.9656 0.1045 
1.6000 2.7482 4.3482 4.3971 0.0989 
1.7000 2.8471 4.5471 4.8400 0.0939 
1.8000 2.9410 4.7410 5.2939 0.0896 
1.9000 3.0306 4.9306 5.7581 0.0856 
2.0000 3.1162 


Xo + to 24+1 


X= Xo + Uf to, Xo) = X9+ HAA = 2+ 0.1 = 2.1500 


Xolo 
ty = t, +h = 1.1000 + 0.1 = 1.2000 


x, +t, = 2.1500 4.0.1 221500 + 1.100 


X,=x,+hf(t ara 
x, + Aft, m) =x ve 2.1500 - 1.100 


= 2.2874 


The rest of the solution is obtained step by step as set out in Figure 2.6. The approxima- 
tion X(2) = 3.1162 results. 


The solution to this example could easily be obtained using MAPLE as follows: 
S Clee saclirt (e(e) eal (ie) ee) 7 (ee) ie) eamiieiea< (il) eae 
Sale —C SOnlaer (iG ooplemmslembinten aia 
numeric,method=classical [foreuler] , output=listprocedure, 
stepsize=0.1); 
> solesoo (2,21 (21) esol (2) ¢ 


Analysing Euler’s method 


We have introduced Euler’s method via an intuitive argument from a geometrical 
understanding of the problem. Euler’s method can be seen in another light — as an 
application of the Taylor series. The Taylor series expansion for a function x(t) gives 
e294 ne =o) + - SH 4 i Pay 4. (2.4) 
3! dt 
Using this formula, we could, in theory, given the value of x(f) and all the derivatives 
of x at tf, compute the value of x(t + A) for any given h. If we choose a small value for 
h then the Taylor series truncated after a finite number of terms will provide a good 
approximation to the value of x(t + h). Euler’s method can be interpreted as using 
the Taylor series truncated after the second term as an approximation to the value of 
x(t +h). 
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In order to distinguish between the exact solution of a differential equation and a 
numerical approximation to the exact solution (and it should be appreciated that all 
numerical solutions, however accurate, are only approximations to the exact solu- 
tion), we shall now make explicit the convention that we used in the last section. The 
exact solution of a differential equation will be denoted by a lower-case letter and a 
numerical approximation to the exact solution by the corresponding capital letter. Thus, 
truncating the Taylor series, we write 


X(t+h) = x(t) + nv) = x(é) + hflt, x) (2.5) 


Applying this truncated Taylor series, starting at the point (¢), x9) and denoting 4 + nh 
by ¢,, we obtain 


X(t.) = X(to + h) = x(to) + hf (to, Xo) 

X(t,) = X(t, +h) = X(t,) + Af(t,, X) 

X(t;) = X(t, + h) = X(b) + Af(b, X) 
and so on 


which is just the Euler-method formula obtained in Section 2.3.1. As an additional 
abbreviated notation, we shall adopt the convention that x(t) + nh) is denoted by x,, 
X(to + nh) by X,, f(t, x,) by f,, and f(t,, X,) by F,. Hence we may express the Euler 
method, in general terms, as the recursive rule 


Xp = Xo 
Xa=X,+hF, (n= 0) 


The advantage of viewing Euler’s method as an application of Taylor series in this way 
is that it gives us a clue to obtaining more accurate methods for the numerical solution 
of differential equations. It also enables us to analyse in more detail how accurate 
the Euler method may be expected to be. Using the order notation we can abbreviate 
(2.4) to 


x(t + h) = x(t) + hf(t, x) + O(R’) 
and, combining this with (2.5), we see that 
X(t + h) =x(t+ h) + O(h’) (2.6) 


(Note that in obtaining this result we have used the fact that signs are irrelevant in 
determining the order of terms; that is, -O(h”) = O(h”).) Equation (2.6) expresses the 
fact that at each step of the Euler process the value of X(t + /) obtained has an error of 
order h’, or, to put it another way, the formula used is accurate as far as terms of order 
h. For this reason Euler’s method is known as a first-order method. The exact size of 
the error is, as we intuitively expected, dependent on the size of h, and decreases as h 
decreases. Since the error is of order h’, we expect that halving A, for instance, will 
reduce the error at each step by a factor of four. 

This does not, unfortunately, mean that the error in the solution of the initial value 
problem is reduced by a factor of four. To understand why this is so, we argue as 
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Example 2.2 


Solution 


Figure 2.7 
Computational results 
for Example 2.2. 


follows. Starting from the point (f), x)) and using Euler’s method with a step size h to 
obtain a value of X(t, + 4), say, requires 4/h steps. At each step an error of order /” is 
incurred. The total error in the value of X(¢) + 4) will be the sum of the errors incurred 
at each step, and so will be 4/h times the value of a typical step error. Hence the total 
error is of the order of (4/h)O(h’); that is, the total error is O(h). From this argument we 
should expect that if we compare solutions of a differential equation obtained using 
Euler’s method with different step sizes, halving the step size will halve the error in the 
solution. Examination of Figure 2.5 confirms that this expectation is roughly correct in 
the case of the solutions presented there. 


Let X, denote the approximation to the solution of the initial-value problem 


2 
dx _ _x 


dt ¢+1° 


x(0) =1 


obtained using Euler’s method with a step size h = 0.1, and_X, that obtained using a step size 
of h = 0.05. Compute the values of X,(f) and_X,(f) for f= 0.1, 0.2,..., 1.0. Compare 
these values with the values of x(t), the exact solution of the problem. Compute the ratio 
of the errors in X, and X,,. 


The exact solution, which may be obtained by separation, is 


Pee eisccrer! ee 
1 — In(t + 1) 


The numerical solutions X, and X, and their errors are shown in Figure 2.7. Of course, 
in this figure the values of X, are recorded at every step whereas those of X, are only 
recorded at alternate steps. 

Again, the final column of Figure 2.7 shows that our expectations about the effects 
of halving the step size when using Euler’s method to solve a differential equation are 
confirmed. The ratio of the errors is not, of course, exactly one-half, because there are 
some higher-order terms in the errors, which we have ignored. 


-X, 
t x, %, x(f) Ix-X,| Ix—X| EsseS 


[x-X,| 

0.000 00 1.000 00 1.000 00 1.000 00 

0.100 00 1.10000 1.102 50 1.105 35 0.005 35 0.002 85 0.53 
0.200 00 1.21000 1.21603 1.22297 0.012 97 0.006 95 0.54 
0.30000 1.33201 1.34294 1.355 68 0.023 67 0.012 75 0.54 
0.400 00 1.468 49 1.486 17 1.507 10 0.038 61 0.020 92 0.54 
0.500 00 1.622 52 1.649 52 1.68199 0.059 47 0.032 47 0.55 
0.600 00 1.798 03 1.83791 1.88681 0.088 78 0.048 90 0.55 
0.700 00 2.000 08 2.057 92 2.13051 0.13042 0.072 59 0.56 
0.800 00 2.235 40 2.31857 2.425 93 0.190 53 0.107 36 0.56 
0.900 00 2.51301 2.63251 2.792 16 0.279 15 0.159 65 O37 
1.000 00 2.845 39 3.01805 3.258 89 0.413 50 0.240 84 0.58 
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2.3.3 


Example 2.3 


Solution 


Using numerical methods to solve engineering problems 


In Example 2.2 the errors in the values of X, and X, are quite large (up to about 14% in 
the worst case). While carrying out computations with large errors such as these is quite 
useful for illustrating the mathematical properties of computational methods, in engineering 
computations we usually need to keep errors very much smaller. Exactly how small they 
must be is largely a matter of engineering judgement. The engineer must decide how 
accurately a result is needed for a given engineering purpose. It is then up to that engineer 
to use the mathematical techniques and knowledge available to carry out the computations 
to the desired accuracy. The engineering decision about the required accuracy will usually 
be based on the use that is to be made of the result. If, for instance, a preliminary design 
study is being carried out then a relatively approximate answer will often suffice, whereas 
for final design work much more accurate answers will normally be required. It must be 
appreciated that demanding greater accuracy than is actually needed for the engineering 
purpose in hand will usually carry a penalty in time, effort or cost. 

Let us imagine that, for the problem posed in Example 2.2, we had decided we needed 
the value of x(1) accurate to 1%. In the cases in which we should normally resort to 
numerical solution we should not have the analytical solution available, so we must 
ignore that solution. We shall suppose then that we had obtained the values of X,(1) and 
X,(1) and wanted to predict the step size we should need to use to obtain a better appro- 
ximation to x(1) accurate to 1%. Knowing that the error in_X,(1) should be approximately 
one-half the error in X,(1) suggests that the error in X,(1) will be roughly the same as 
the difference between the errors in X,(1) and _X,(1), which is the same as the difference 
between X,(1) and _X,(1); that is, 0.172 66. One per cent of X,(1) is roughly 0.03, that is 
roughly one-sixth of the error in X,(1). Hence we expect that a step size roughly one- 
sixth of that used to obtain X, will suffice; that is, a step size h = 0.008 33. In practice, 
of course, we shall round to a more convenient non-recurring decimal quantity such as 
h=0.008. This procedure is closely related to the Aitken extrapolation procedure some- 
times used for estimating limits of convergent sequences and series. 


Compute an approximation X(1) to the value of x(1) satisfying the initial-value problem 


dx x 4 
a eit 


by using Euler’s method with a step size A = 0.008. 


It is worth commenting here that the calculations performed in Example 2.2 could 
reasonably be carried out on any hand-held calculator, but this new calculation requires 
125 steps. To do this is on the boundaries of what might reasonably be done on a hand- 
held calculator, and is more suited to a micro- or minicomputer. Repeating the calcula- 
tion with a step size h = 0.008 produces the result X(1) = 3.21391. 

We had estimated from the evidence available (that is, values of X(1) obtained using 
step sizes h = 0.1 and 0.05) that the step size / = 0.008 should provide a value of X(1) 
accurate to approximately 1%. Comparison of the value we have just computed with the 
exact solution shows that it is actually in error by approximately 1.4%. This does not quite 
meet the target of 1% that we set ourselves. This example therefore serves, first, to illustrate 
how, given two approximations to x(1) derived using Euler’s method with different step 
sizes, we can estimate the step size needed to compute an approximation within a 
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desired accuracy, and, secondly, to emphasize that the estimate of the appropriate step 
size is only an estimate, and will not guarantee an approximate solution to the problem 
meeting the desired accuracy criterion. If we had been more conservative and rounded 
the estimated step size down to, say, 0.005, we should have obtained X(1) = 3.23043, 
which is in error by only 0.9% and would have met the required accuracy criterion. 


(2) Again the solution to this example could be obtained using MAPLE. The following 
worksheet computes the numerical solution using a step size of 0.008, then the 
analytical solution and finally computes the percentage error in the numerical solution. 


#Set up differential equation 

Glecil s =clauitié (Gz (ie)) 1c axe (ie) S27) (eae) p abate lp Sse (10) IL 

#obtain xl, the numerical solution 

x1l:=dsolve({deql, initl1}, 
numeric,method=classical [foreuler] , output=listprocedure, 
stepsize=0.008); 


We WE WY SY 


#xa is the analytic solution 
xa:=dsolve({deql, initl1}); 
#obtain the value of x(t) at t=1 
CO (2,2 (21) (CL) 3 
#find the percentag rror in the numerical solution 
eval£((op(2,x1[2]) (1) -subs (t=1,0p(2,xa)))/ 
Sulos (ie=i1l , om (2 ,2<2)) )) ) = LOW? 


N/a VV VV 


Since we have mentioned in Example 2.3 the use of computers to undertake the 
repetitive calculations involved in the numerical solution of differential equations, it is 
also worth commenting briefly on the writing of computer programs to implement those 
numerical solution methods. Whilst it is perfectly possible to write informal, unstruc- 
tured programs to implement algorithms such as Euler’s method, a little attention to 
planning and structuring a program well will usually be amply rewarded — particularly 
in terms of the reduced probability of introducing ‘bugs’. Another reason for careful 
structuring is that, in this way, parts of programs can often be written in fairly general 
terms and can be re-used later for other problems. The two pseudocode algorithms in 
Figures 2.8 and 2.9 will both produce the table of results in Example 2.2. The pseudocode 
program of Figure 2.8 is very specific to the problem posed, whereas that of Figure 2.9 
is more general, better structured, and more expressive of the structure of mathematical 
problems. It is generally better to aim at the style of Figure 2.9. 


Figure 2.8 A poorly 
structured algorithm 
for Example 2.2. 


xlel 
x21 
write(vdu, 0, 1, 1, 1) 
for iis | to 10 do 
xl — xl + 0.1*x1l*x1/((i-1)*0.1 + 1) 
x2 — x2 + 0.05*x2*x2/((i-1)*0.1 + 1) 
x2 — x2 + 0.05*x2*x2/((i-1)*0.1 + 1.05) 
x © 1/(1 — InG+0.1 + 1)) 
write(vdu,0. 1 *i,x1,x2,x,x — x1,x — x2,(x — x2)/(x — xl)) 
endfor 
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Figure 2.9 A better 


structured algorithm 
for Example 2.2. 


initial_time < 0 

final_time < 1 

initial_x < 1 

step <— 0.1 

t < initial_time 

xl < initial_x 

x2 < initial_x 

hl < step 

h2 < step/2 

write(vdu,initial_time,x1,x2,initial_x) 

repeat 
euler(t,x1,h1,1 — x1) 
euler(t,x2,h2,2 > x2) 
t<t-+ step 


127 


x © exact_solution(t,initial_time,initial_x) 
write(vdu,t,x1,x2,x,abs(x — x1),abs(x — x2),abs((x — x2)/(x — x1))) 


until t > final_time 


procedure euler(t_old,x_old,step,number — x_new) 


temp_x < x_old 
for i is 0 to number —1 do 


temp_x < temp_x + step*derivative(t_old + step*i,temp_x) 


endfor 
X_new <— temp_x 
endprocedure 


procedure derivative(t,x — derivative) 
derivative — x*x/(t + 1) 
endprocedure 


procedure exact_solution(t,t0,x0 — exact_solution) 


c & In(t0 + 1) + 1/x0 
exact_solution — I/(c — In(t + 1)) 
endprocedure 


2.3.4 Exercises 


All the exercises in this section can be completed using MAPLE ina similar manner to Examples 2.1 and 2.3 above. 
In particular MAPLE or some other form of computer assistance should be used for Exercises 5, 6 and 7. If you do 
not have access to MAPLE, you will need to write a program in MATLAB or some other high-level scientific 


computer programming language (e.g. Pascal or C). 

Find the value of X(0.3) for the initial-value problem 5 
x(0) = 1 

using Euler’s method with step size h = 0.1. 


Find the value of X(1.1) for the initial-value problem 4 


using Euler’s method with step size h = 0.025. 
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Find the value of X(1) for the initial-value problem 


dx x 


a X41)’ x(0.5) = 1 


using Euler’s method with step size h = 0.1. 


Find the value of X(0.5) for the initial-value problem 


dx _4-1 


a =] 
dt t+x’ a 


using Euler’s method with step size h = 0.05. 
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Denote the Euler-method solution of the initial- 
value problem 


dx _ _ xt 


= > waCljy=2 
dt f¢f+2 


using step size h = 0.1 by X,(4), and that using 
h=0.05 by X,(2). Find the values of X,(2) and 
X,(2). Estimate the error in the value of X,(2), and 
suggest a value of step size that would provide a 
value of X(2) accurate to 0.1%. Find the value of 
X(2) using this step size. Find the exact solution of 
the initial-value problem, and determine the actual 
magnitude of the errors in_X,(2), X,(2) and your 
final value of X(2). 


Denote the Euler-method solution of the initial- 
value problem 


dx_ 1 

ie 1l)=1 

dt xt at) 
using step size h = 0.1 by X,(4), and that using 
h=0.05 by X,(f). Find the values of X,(2) and 


2.3.5 


X,(2). Estimate the error in the value of X,(2), and 
suggest a value of step size that would provide a 
value of X(2) accurate to 0.2%. Find the value of 
X(2) using this step size. Find the exact solution of 
the initial-value problem, and determine the actual 
magnitude of the errors in_X,(2), X,(2) and your 
final value of X(2). 


Denote the Euler-method solution of the initial- 
value problem 


using step size h = 0.05 by_X,(¢), and that using 
h=0.025 by X,(¢). Find the values of X,(1.5) and 
X,(1.5). Estimate the error in the value of X,(1.5), 
and suggest a value of step size that would provide 
a value of X(1.5) accurate to 0.25%. Find the value 
of X(1.5) using this step size. Find the exact 
solution of the initial-value problem, and determine 
the actual magnitude of the errors in_X,(1.5), X,(1.5) 
and your final value of X(1.5). 


More accurate solution methods: multistep methods 


In Section 2.3.2 we discovered that using Euler’s method to solve a differential equa- 
tion is essentially equivalent to using a Taylor series expansion of a function truncated 
after two terms. Since, by so doing, we are ignoring terms O(h’), an error of this order 
is introduced at each step in the solution. Could we not derive a method for calculat- 
ing approximate solutions of differential equations which, by using more terms of the 
Taylor series, provides greater accuracy than Euler’s method? We can — but there are 
some disadvantages in so doing, and various methods have to be used to overcome 


these. 


Let us first consider a Taylor series expansion with the first three terms written 


explicitly. This gives 


xt+h=x0+ 820)+ : 


dt 


Substituting f(¢, x) for dx/dt, we obtain 


x(t+ h) = x(t) + Af(t, x) + 


(t) + O(h*) (2.7) 


H dfey x) + OM) 


Dropping the O(h*) terms provides an approximation 


X(t+h) =x(t) + f(t.) + " si t,x) 


such that 


X(t +h) =x(t + h) + O(n’) 
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in other words, a numerical approximation method which has an error at each step that 
is not of order h’ like the Euler method but rather of order h. The corresponding general 
numerical scheme is 


h’ dF, 


Xin =X, + hE + = 
2 dt 


: (2.8) 

The application of the formula (2.5) in Euler’s method was straightforward because 
an expression for f(t, x) was provided by the differential equation itself. To apply (2.8) 
as it stands requires an analytical expression for df/dt so that dF./dt may be computed. 
This may be relatively straightforward to provide — or it may be quite complicated. 
Although, using modern computer algebra systems, it is now often possible to compute 
analytical expressions for the derivatives of many functions, the need to do so remains 
a considerable disadvantage when compared with methods which do not require the 
function’s derivatives to be provided. 

Fortunately, there are ways to work around this difficulty. One such method hinges 
on the observation that it is just as valid to write down Taylor series expansions for 
negative increments as for positive ones. The rch series expansion of x(t — h) is 
he - x 


fF fay 
3! dt 


x(t — h) = x(t) - dx 4) +— 
is 
If we write only the first three terms explicitly, we have 
x(t h) =x(8)- mm ip ie fx + OW) 
or, rearranging the equation, 


he dx 
2!d 


ile x(t — Ah) — x(t) + ne =) + O(h*) 
Substituting this into (2.7), we obtain 
x(t-+ h) = x(t) + nr) +leG=h) =k) = nr) + O(h)| + O08) 
That is, 
x(t+h)=x(t-—h)+ 2neX(t) + O(h’) 
or, substituting f(¢, x) for dx/dt, 


x(t-+ h) =x(t— h) + 2hf(t, x) + O(h) (2.9) 


Alternatively, we could write down the Taylor series expansion of the function dx/dt 
with an increment of —/A: 


2 2 43 
8x — py = By) — 2 Oy + 2 Ry - O10) 
dt 7 dt 2! dt 


Writing only the first two terms explicitly and rearranging gives 


nF = 2 — 2G _ py 4 OF) 
dt dt dt 
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and substituting this into (2.4) gives 
Sisco Oy Th Up Aon. 
x(t + h) = x(t) + ha) + 5 Po att h)+ O(h ) + O(h’) 
That is, 
Mts W=x04 5H = Se S | + Of) 
or, substituting /(¢, x) for dx/dt, 
x(t +h) = x(t) + 5h[3f(t, (0) —f(t— A, x(t — h))] + OCH’) (2.10) 
Equations (2.7), (2.9) and (2.10) each give an expression for x(f + A) in which all 
terms up to those in h* have been made explicit. In the same way as, by ignoring terms 


of O(h*) in (2.7), the numerical scheme (2.8) can be obtained, (2.9) and (2.10) give rise 
to the numerical schemes 


Xn = X,-1 + 2hF, (2.11) 
and 
Xu =X, + FAB, — F4) (2.12) 


respectively. Each of these alternative schemes, like (2.8), incurs an error O(h*) at 
each step. 

The advantage of (2.11) or (2.12) over (2.8) arises because the derivative of 
f(t, x) in (2.7) has been replaced in (2.9) by the value of the function x at the 
previous time, x(t — /), and in (2.10) by the value of the function fat time t — h. This 
is reflected in (2.11) and (2.12) by the presence of the terms in X,,_, and F,,_, respect- 
ively and the absence of the term in dF,/dt. The elimination of the derivative of the 
function f(t, x) from the numerical scheme is an advantage, but it is not without its 
penalties. In both (2.11) and (2.12) the value of X,,, depends not only on the values 
of X, and F’, but also on the value of one or the other at ¢,_,. This is chiefly a problem 
when starting the computation. In the case of the Euler scheme the first step took 
the form 


X, =X, +hF, 

In the case of (2.11) and (2.12) the first step would seem to take the forms 
X, =X_, + 2hFy 

and 
X, =X) + ABE, — F) 


respectively. The value of X_, in the first case and F_, in the second is not normally 
available. The resolution of this difficulty is usually to use some other method to 
start the computation, and, when the value of X,, and therefore also the value of F), 
is available, change to (2.11) or (2.12). The first step using (2.11) or (2.12) therefore 
involves 
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Example 2.4 


Solution 


Figure 2.10 
Computational results 
for Example 2.4. 


X, =X) + DAF, 
or 
X,=X,+ LAGF, — Fy) 


Methods like (2.11) and (2.12) that involve the values of the dependent variable or its 
derivative at more than one value of the independent variable are called multistep 
methods. These all share the problem that we have just noted of difficulties in deciding 
how to start the computation. We shall return to this problem of starting multistep methods 
in Section 2.3.7. 


Solve the initial-value problem 
es 
dt t+1’ 


x(0) = 1 


posed in Example 2.2 using the scheme (2.12) with a step size h = 0.1. Compute the 
values of X(f) for f= 0.1, 0.2,..., 1.0 and compare them with the values of the exact 
solution x(f). 


We shall assume that the value of X(0.1) has been computed using some other method 
and has been found to be 1.10535. The computation therefore starts with the calculation 
of the values of F, Ff) and hence X,. Using the standard notation we have t)= 0, and 
Xy= 1. The function f(t, x) =x*/(t+ 1). Using the given value X(0.1) = 1.105 35, we have 
t, = 0.1, and X, = 1.10535. So the first step is 


t= t, +h=0.100 00 + 0.1 = 0.200 00 
X, =X, + SAGBF, —fy)=X, + SAL3f(t, Xi) — f(t, Xo)] 


Xt Xo 


=X,+$0(3 
fit+1 ft4+1 


2 2 
} = 1.10535 + 0.1( 3tanene = zt] = 1.22196 
O1+1 OF1 


The results of the computation are shown in Figure 2.10. 


t x F, LAGE, — Fy) x(f) Ix -X,| 
0.000 00 1.000 00 1.000 00 

0.100 00 1.105 35 1.11073 0.11661 1.105 35 0.000 00 
0.200 00 1.22196 1.244 32 0.13111 1.22297 0.001 01 
0.300 00 1.353 07 1.408 31 0.149 03 1.355 68 0.002 61 
0.400 00 1.502 10 1.611 65 0.17133 1.507 10 0.004 99 
0.500 00 1.673 44 1.866 92 0.199 46 1.68199 0.008 55 
0.600 00 1.872 89 2.19233 0.235 50 1.88681 0.01391 
0.700 00 2.108 39 2.61490 0.282 62 2.13051 0.022 11 
0.800 00 2.39101 3.17608 0.345 67 2.425 93 0.034 92 
0.900 00 2.736 68 3.941 80 0.432 47 2.792 16 0.055 48 
1.000 00 3.169 14 3.258 89 0.089 75 
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Example 2.5 


Solution 


It is instructive to compare the values of XY computed in Example 2.4 with those com- 
puted in Example 2.2. Since the method we are using here is a second-order method, 
the error at each step should be O(h*) rather than the O(h’) error of the Euler method. 
We are using the same step size as for the solution X, of Example 2.2, so the errors 
should be correspondingly smaller. Because in this case we know the exact solution of 
the differential equation, we can compute the errors. Examination of the results shows 
that they are indeed much smaller than those of the Euler method, and also considerably 
smaller than the errors in the Euler method solution X, which used step size h = 0.05, 
half the step size used here. 

In fact, some numerical experimentation (which we shall not describe in detail) 
reveals that to achieve a similarly low level of errors, the Euler method requires a step 
size h = 0.016, and therefore 63 steps are required to find the value of X(1). The second- 
order method of (2.12) requires only 10 steps to find X(1) to a similar accuracy. Thus 
the solution of a problem to a given accuracy using a second-order method can be 
achieved in a much shorter computer processing time than using a first-order method. 
When very large calculations are involved or simple calculations are repeated very 
many times, such savings are very important. 

How do we choose between methods of equal accuracy such as (2.11) and (2.12)? 
Numerical methods for the solution of differential equations have other properties 
apart from accuracy. One important property is stability. Some methods have the 
ability to introduce gross errors into the numerical approximation to the exact solu- 
tion of a problem. The sources of these gross errors are the so-called parasitic 
solutions of the numerical process, which do not correspond to solutions of the 
differential equation. The analysis of this behaviour is beyond the scope of this 
book, but methods that are susceptible to it are intrinsically less useful than those 
that are not. The method of (2.11) can show unstable behaviour, as demonstrated in 
Example 2.5. 


Let X, denote the approximation to the solution of the initial-value problem 


qx 374964, x(0)=2 

dt 
obtained using the method defined by (2.11), and X, that obtained using the method 
defined by (2.12), both with step size h = 0.1. Compute the values of X,(¢) and_X,(¢) for 
t=0.1, 0.2, ..., 2.0. Compare these with the values of x(f), the exact solution of the 
problem. In order to overcome the difficulty of starting the processes, assume that the 
value X(0.1) = 1.645 66 has been obtained by another method. 


The exact solution of the problem, which is a linear equation and so may be solved by 
the integrating-factor method, is 


x=e*+e™ 


The numerical solutions X, and X, and their errors are shown in Figure 2.11. It can be 
seen that X, exhibits an unexpected oscillatory behaviour, leading to large errors in the 
solution. This is typical of the type of instability from which the scheme (2.11) and 
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Figure 2.11 
Computational results 
for Example 2.5. 


t XxX, X, x(t) x—-X, x-X, 
0.000 00 2.000 00 2.000 00 2.000 00 

0.100 00 1.645 66 1.645 66 1.645 66 0.000 00 0.000 00 
0.200 00 1.37454 1.376 56 1.367 54 —0.007 00 —0.009 02 
0.300 00 1.148 42 1.15909 1.14739 —0.001 04 —0.011 70 
0.400 00 0.981 82 0.984 36 0.97151 —0.010 30 —0.012 84 
0.500 00 0.827 46 0.842 27 0.829 66 0.002 20 0.01261 
0.600 00 0.72795 0.725 83 0.714 11 —0.013 84 0.01172 
0.700 00 0.610 22 0.629 54 0.619 04 0.008 83 —0.010 50 
0.800 00 0.560 45 0.549 22 0.540 05 —0.020 41 —0.009 17 
0.900 00 0.453 68 0.481 64 0.473 78 0.020 10 —0.007 86 
1.000 00 0.450 88 0.424 32 0.417 67 —0.033 21 —0.006 66 
1.100 00 0.330 30 0.375 33 0.369 75 0.039 45 —0.005 58 
1.200 00 0.385 84 0.333 15 0.328 52 —0.057 33 —0.004 64 
1.300 00 0.21927 0.296 60 0.292 77 0.073 50 —0.003 83 
1.400 00 0.363 29 0.264 75 0.26159 —0.101 70 —0.003 15 
1.500 00 0.099 93 0.236 83 0.234 24 0.13431 —0.002 59 
1.600 00 0.392 59 0.21225 0.210 13 —0.182 46 —0.002 12 
1.700 00 —0.054 86 0.190 52 0.188 78 0.243 64 —0.001 73 
1.800 00 0.498 57 0.17124 0.169 82 —0.328 76 —0.001 42 
1.900 00 —0.287 88 0.15408 0.15291 0.440 80 —0.001 16 
2.000 00 0.731 13 0.138 77 0.13781 —0.593 32 —0.000 96 


those like it are known to suffer. The scheme defined by (2.11) is not unstable for all 
differential equations, but only for a certain class. The possibility of instability in 
numerical schemes is one that should always be borne in mind, and the intelligent user 
is always critical of the results of numerical work and alert for signs of this type of 
problem. 


In this section we have seen how, starting from the Taylor series for a function, 
schemes of a higher order of accuracy than Euler’s method can be constructed. We have 
constructed two second-order schemes. The principle of this technique can be extended 
to produce schemes of yet higher orders. They will obviously introduce more values 
of X,, or F,, (where m=n—2,n—3,...). The scheme (2.12) is, in fact, a member of a 
family of schemes known as the Adams—Bashforth formulae. The first few members 
of this family are 


Xu =X, + AF, 

X,1 =X, + GF, - F,.) 

Neg = ep h(23h — 16M a oF) 

Xn =X, + £WGSF, — 59F_, + 37F.— 9F.5) 


The formulae represent first-, second-, third- and fourth-order methods respectively. The 
first-order Adams—Bashforth formula is just the Euler method, the second-order 
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2.3.6 


one is the scheme we introduced as (2.12), while the third- and fourth-order formulae 
are extensions of the principle we have just introduced. Obviously all of these require 
special methods to start the process in the absence of values of X_,, F_,, X_,, F_, and 
so on. 

Some of the methods used by the standard MATLAB procedures for numerical 
solution of ODEs are based on more sophisticated versions of the multistep methods 
which we have just introduced. Multistep methods are particularly suitable for solving 
equations in which the derivative function, f(t, x), is relatively computationally costly 
to evaluate. At each step a multistep methods can reuse the values of the function 
already computed at previous steps so the number of evaluations of the derivative 
function is reduced compared to some other methods. 


Local and global truncation errors 


In Section 2.3.2 we argued intuitively that, although the Euler method introduces an 
error O(h’) at each step, it yields an O(A) error in the value of the dependent variable 
corresponding to a given value of the independent variable. What is the equivalent 
result for the second-order methods we have introduced in Section 2.3.5? We shall 
answer this question with a slightly more general analysis that will also be useful to us 
in succeeding sections. 

First let us define two types of error. The local error in a method for integrating 
a differential equation is the error introduced at each step. Thus if the method is 
defined by 


Xai = g(h bo Ae Xi» aa ) 


> "no “*n9 “n-1? 
and analysis shows us that 


Xap = Ay bys Xs bys Nps «+» ) + O(H?*) 


ono Sn nl n—12 


then we say that the local error in the method is of order p + | or that the method is a 
pth-order method. 

The global error of an integration method is the error in the value of X(t) + a) 
obtained by using that method to advance the required number of steps from a known 
value of x(t)). Using a pth-order method, the first step introduces an error O(h?*'). The 
next step takes the approximation X, and derives an estimate X, of x, that introduces a 
further error O(h”*'). The number of steps needed to calculate the value X(t, + a) is a/h. 
Hence we have 


X(t + a) = x(ty + a) + ; O(h?*') 


Dividing a quantity that is O(h’) by h produces a quantity that is O(h"'), so we must 
have 


X(ty + a) = x(t) + a) + O(h’) 


In other words, the global error produced by a method that has a local error O(h?*') 
is O(h’). As we saw in Example 2.2, halving the step size for a calculation using 
Euler’s method produces errors that are roughly half as big. This is consistent with 
the global error being O(/). Since the local error of the Euler method is O(h°), this is 
as we should expect. Let us now repeat Example 2.2 using the second-order Adams— 
Bashforth method, (2.12). 
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Example 2.6 


Solution 


Figure 2.12 
Computational results 
for Example 2.6. 


Let X, denote the approximation to the solution of the initial-value problem 


dx _ x 0)=1 
dt t+1’ any 


obtained using the second-order Adams—Bashforth method with a step size h = 0.1, and 
X, that obtained using a step size of h = 0.05. Compute the values of X,(f) and_X,(¢) for 
t=0.1, 0.2,..., 1.0. Compare these values with the values of x(f), the exact solution 
of the problem. Compute the ratio of the errors in X, and X,. In order to start the process, 
assume that the values X(—0.1) = 0.904 68 and X(—0.05) = 0.951 21 have already been 
obtained by another method. 


The exact solution was given in Example 2.2. The numerical solutions X, and X, and 
their errors are shown in Figure 2.12. 

Because the method is second-order, we expect the global error to vary like h’. 
Theoretically, then, the error in the solution X, should be one-quarter that in_X,. We see 
that this expectation is approximately borne out in practice. 


[x- 4] 
t x X x(0) Ix-X Ix—X| 


Xx ~ Ap 

|z-X,| 
0.000 00 1.000 00 1.000 00 1.000 00 
0.100 00 1.104 53 1.105 12 1.105 35 0.000 82 0.000 23 0.28 
0.200 00 1.220 89 1.222 39 1.222 97 0.002 08 0.000 58 0.28 
0.300 00 1.35176 1.35459 1.355 68 0.003 92 0.001 09 0.28 
0.400 00 1.500 49 1.505 25 1.507 10 0.006 61 0.001 85 0.28 
0.500 00 1.671 44 1.679 03 1.68199 0.01055 0.002 96 0.28 
0.600 00 1.870 40 1.882 17 1.886 81 0.016 40 0.004 64 0.28 
0.700 00 2.105 25 2.123 31 2.13051 0.025 25 0.007 20 0.29 
0.800 00 2.387 00 2.41470 2.425 93 0.038 93 0.011 23 0.29 
0.900 00 2.73145 2.77440 2.792 16 0.060 70 0.017 76 0.29 
1.000 00 3.162 20 3.23007 3.258 89 0.096 70 0.028 82 0.30 


Just as previously we outlined how, for the Euler method, we could estimate from 
two solutions of the differential equation the step size that would suffice to compute a 
solution to any required accuracy, so we can do the same in a more general way. If we 
use a pth-order method to compute two estimates X,(¢) + a) and X,(t) + a) of x(t) + a) 
using step sizes h and th then, because the global error of the process is O(h’), we 
expect the error in X,(f) + a) to be roughly 2” times that in X,(¢) + a). Hence the error in 
X, (to + a) may be estimated to be 


|X, (to + a) — X,(to + a) | 
2?-) 
If the desired error, which may be expressed in absolute terms or may be derived from 


a desired maximum percentage error, is € then the factor k, say, by which the error in 
X(t) + a) must be reduced is 


bie |X,(t) + a) — X,(to + 2) | 
e(2’- 1) 
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Example 2.7 


Solution 


2.3.7 


Since reducing the step size by a factor of qg will, for a pth-order error, reduce the error 
by a factor of g’, the factor by which step size must be reduced in order to meet the 
error criterion is the pth root of k. The step size used to compute X, is th, so finally we 
estimate the required step size as 


| eae Cs Vee! 
5 ee (2.13) 


This technique of estimating the error in a numerical approximation of an unknown 
quantity by comparing two approximations of that unknown quantity whose order of 
accuracy is known is an example of the application of Richardson extrapolation. 


Estimate the step size required to compute an estimate of x(1) accurate to 2dp for the 
initial-value problem in Example 2.6 given the values X,(1) = 3.16220 and X,(1) = 
3.230 07 obtained using step sizes h = 0.1 and 0.05 respectively. 


For the result to be accurate to 2dp the error must be less than 0.005. The estimates 
X,(1) and X,(1) were obtained using a second-order process, so, applying (2.13), with 
€= 0.005, $4 = 0.05 and p = 2, we have 


0.015 


h=0.05 (ee 
[3.162 20 - 3.230 07| 


1/2 
} = 0.0235 
In a real engineering problem what we would usually do is round this down to say 
0.02 and recompute X(1) using step sizes h = 0.04 and 0.02. These two new estimates 
of X(1) could then be used to estimate again the error in the value of X(1) and confirm 
that the desired error criterion had been met. 


More accurate solution methods: predictor—corrector 
methods 


In Section 2.3.5 we showed how the third term in the Taylor series expansion 
2 42 

x(t hy=x() + hy +E 
dt 2! dt 

could be replaced by either x(t — /) or (dx/dt)(t— h). These are not the only possibilities. 

By using appropriate Taylor series expansions, we could replace the term with other values 
of x(t) or dx/dt. For instance, expanding the function x(t — 2h) about x(f) gives rise to 


(t) + O(h’) (2.14) 


2 
2d°x 


—(t)+ O(n’) (2.15) 
dt 


x(t— 2h) =x() — 2h 222) 42h 
dt 
and eliminating the second-derivative term between (2.14) and (2.15) gives 


x(t-+ A) = 2 x(t) +b x(t 2h) +3h (0 + O(F°) 
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which, in turn, would give rise to the integration scheme 
Xn = 3X, + 1X,-2 + 3 hF, 


Such a scheme, however, would not seem to offer any advantages to compensate for 
the added difficulties caused by a two-step scheme using non-consecutive values of X. 

The one alternative possibility that does offer some gains is using the value of 
(dx/dt)(t + A). Writing the Taylor series expansion of (dx/dt)(t + A) yields 


d°x 


=(t) + O(h’) 
dt 


Ges 2) = er) +h 
dt dt 
and eliminating the second derivative between this and (2.14) gives 
h| dx dx 3 
h)= -|— = h h 2.16 
x(t + Ah) = x(t) + 4/889 + aos ] + O(h’) ( ) 


leading to the integration scheme 
Xue = Xx, at TAF, si Fis) (2.17) 


This, like (2.11) and (2.12), is a second-order scheme. It has the problem that, in order 
to calculate X,,,,, the value of F,,,; 1s needed, which, in its turn, requires that the value 
of X,,,, be known. This seems to be a circular argument! 

One way to work around this problem and turn (2.17) into a usable scheme is to start 
by working out a rough value of X,,,,, use that to compute a value of F,,,,;, and then use 
(2.17) to compute a more accurate value of X,,;. Such a process can be derived as 
follows. We know that 


x(t+h)=x(6) +h = (1) + OUP) 
Let 
X(t+h) = x(t) +h 2 (0) (2.18) 


then 
x(t + h) = X(t + h) + O(h’) 

or, using the subscript notation defined above, 
Xnst = Xne1 + O(N’) 

Thus 


AX = 


dt 
=f (tists Kets + O(h’)) 


Kt Xn) 


= f(tyats Eyer) + onal (ysis pat) + OCF") 


=f (tris Se) + OC’) (2.19) 
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In the subscript notation (2.16) is 
Xs = Xn t FAC Lbs Xn) + fbnsas Xue) + OCH’) 
Substituting (2.19) into this gives 
Xnet =Xq + 5A bes Xn) + Alu Xns1) + OF) + OCF’) 
That is, 
Xs = Xn t 5A L(t Xn) + Aner» Xue) + OCA’) (2.20) 


Equation (2.20) together with (2.18) forms the basis of what is known as a predictor— 
corrector method, which is defined by the following scheme: 


(1) compute the ‘predicted’ value of X,,,, call it_X,,,, from 
Xie = Xp + Mf ty Xp) (2.21a) 
(2) compute the ‘corrected’ value of X,,, from 
Xr =X + 5M Ll Xn) + frets Xnet)) (2.21b) 
This predictor—corrector scheme, as demonstrated by (2.20), is a second-order method. 
It has the advantage over (2.11) and (2.12) of requiring only the value of X,,, not _X,,_; or 


F,,_;. On the other hand, each step requires two evaluations of the function f(t, x), and 
so the method is less efficient computationally. 


Example 2.8 Solve the initial-value problem 


dx _ x 
dt t+l 


, x(0)=1 


posed in Example 2.2 using the second-order predictor—corrector scheme with a step 
size h = 0.1. Compute the values of X(f) for t= 0.1, 0.2, ..., 1.0 and compare them 
with the values of the exact solution x(f). 


Solution The exact solution was given in Example 2.2. In this example the initial value of ¢ is 
0 and x(0) = 1. Using the standard notation we have t)= 0, and x)= x(f)) = x(0) = 1. 
The function f(t, x) = x*/(t+ 1). So the first two steps of the computation are thus 


2 2. 
; 1 
X, =X + Mf (tos Xp) = Xo + a = 140.1 = 1.100 00 


2 2 
Xo + = 


X, =X + GALA (lo, Xo) +f, Xi] = Xo + (7 +1 +1 


2 A 
= 1,000 00 + j0.1{ + eo = 1.105 00 
2°"lo+1  0.10000+1 
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Figure 2.13 
Computational results 
for Example 2.8. 


EY 


2, 
ee en eae ee ee 
tf, +1 
2 
= 1,105 00 + 0,1 —L105-00_ _ 1 216 00 
0.10000 + 1 


X= X) + SALS(t xX) t+ f(b, X,)] 


2 2 
=X, +1h{“b +2) 
-"\t41 b4+1 
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2 2 
= 1.105 00+ $0.1( tos 00 + als = (09 ti 
0.10000+1 0.20000+1 

The complete computation is set out in Figure 2.13. 

t x, Ltr Xn) Knut I (bers Xn) x(t) Ix —X)| 
0.00000 1.000 00 1.00000 1.10000 1.10000 1.00000 0.00000 
0.10000 1.10500 1.11002 1.21600 1.232 22 1.10535 0.000 35 
0.20000 1.22211 1.24463 1.34658 1.394 82 1.22297 0.000 86 
0.30000 1.35408 1.41042 1.495 13 1.596 72 1.35568 0.001 60 
0.40000 1.50444 1.61667 1.666 11 1.85061 1.507 10 0.002 65 
0.500 00 1.67781 1.876 69 1.86547 2.17500 1.681 99 0.004 18 
0.60000 1.88039 2.209 92 2.10138 2.597 53 1.886 81 0.006 42 
0.70000 2.120 76 2.645 67 2.385 33 3.161 00 2.13051 0.009 75 
0.80000 2.41110 3.229 66 2.734 06 3.934 26 2.42593 0.014 83 
0.90000 2.769 29 4.036 30 3.17292 5.033 72 2.792 16 0.022 87 
1.00000 3.22279 3.258 89 0.036 10 


Again the solution to this example can be obtained using MAPLE. The following 
worksheet computes the numerical and analytical solutions and compares them at 


the required points. 
> #set up differential equation 
S ¢leciloscliirie (G2(ie) ic) eee) 2/7 (eeil) paimnicils=s< (0) =alg 
> #obtain x1, the numerical solution 
= sleacsolwea(iceel, aimed} , 


numeric,method=classical [heunform] , output=listprocedure, 


#xa is the analytic solution 
selcaCsolivea(iclachl, aimaieil}))) 3 

#compute values at required solution points 
HO a icem i ice iO clo 


Wi WOE SNF 


stepsize=0.1); 


i s=0) a seo (2 , Sil || 21))) (ie) ,eavaulit (Ga (2 5221) )) Siac! Clo 


Comparison of the result of Example 2.8 with those of Examples 2.2 and 2.6 shows 
that, as we should expect, the predictor—corrector scheme produces results of consider- 
ably higher accuracy than the Euler method and of comparable (though slightly better) 
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Example 2.9 


Solution 


Figure 2.14 
Computational results 
for Example 2.9. 


accuracy to the second-order Adams—Bashforth scheme. We also expect the scheme to 
have a global error O(h’), and, in the spirit of Examples 2.2 and 2.6, we confirm this in 
Example 2.9. 


Let XY, denote the approximation to the solution of the initial-value problem 


2 
dx _ _x 


dt t+ 


x(0) = 1 


obtained using the second-order predictor—corrector method with a step size h = 0.1, and 
X, that obtained using 4 = 0.05. Compute the values of X,(4) and_X,(4) for t= 0.1, 0.2, ..., 
1.0. Compare these with the values of x(f), the exact solution of the problem. Compute 
the ratio of the errors in X, and X,. 


The numerical solutions X, and X, and their errors are shown in Figure 2.14. The ratio 
of the errors confirms that the error behaves roughly as O(h’). 


t x, x x(t) Ir — Xi Ie XI ea 
4 
0.000 00 1.000 00 1.000 00 1.000 00 
0.10000 1.105 00 1.105 26 1.105 35 0.000 35 0.000 09 0.27 
0.200 00 1.222 11 1.222 74 1.22297 0.000 86 0.000 23 0.27 
0.300 00 1.35408 1.355 25 1.355 68 0.001 60 0.000 43 0.27 
0.400 00 1.504 44 1.506 38 1.507 10 0.002 65 0.000 72 0.27 
0.500 00 1.677 81 1.680 86 1.68199 0.004 18 0.001 13 0.27 
0.600 00 1.880 39 1.885 07 1.886 81 0.006 42 0.001 73 0.27 
0.700 00 2.120 76 2.127 87 2.13051 0.009 75 0.002 64 0.27 
0.800 00 2.41110 2.42190 2.425 93 0.014 83 0.004 03 0.27 
0.900 00 2.769 29 2.785 92 2.792 16 0.022 87 0.006 24 0.27 
1.000 00 3.22279 3.248 98 3.258 89 0.036 10 0.009 91 0.27 


In Section 2.3.5 we mentioned the difficulties that multistep methods introduce 
with respect to starting the computation. We now have a second-order method that 
does not need values of X,_; or earlier. Obviously we can use this method just as 
it stands, but we then pay the penalty, in computer processing time, of the extra 
evaluation of f(t, x) at each step of the process. An alternative scheme is to use the 
second-order predictor—corrector for the first step and then, because the appropriate 
function values are now available, change to the second-order Adams—Bashforth 
scheme — or even, if the problem is one for which the scheme given by (2.11) (which 
is called the central difference scheme) is stable, to that process. In this way we create 
a hybrid process that retains the O(h’) convergence and simultaneously minimizes the 
computational load. 

The principles by which we derive (2.16) and so the integration scheme (2.17) can 
be extended to produce higher-order schemes. Such schemes are called the Adams— 
Moulton formulae and are as follows: 
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2.3.8 


Figure 2.15 

A geometrical 
interpretation of 
the second-order 
predictor—corrector 
method. 


Xia =X, ar nF a 
xe eX SAPs ap ie)) 


er =X, AP + ASF AF Sie fd ia) 


Xu =X, + KAOF 4, + 19F, — 5F,, + F 0) 


nv 


These are first-, second-, third- and fourth-order formulae respectively. They are all like 
the one we derived in this section in that the value of F’,,, is required in order to compute 
the value of X,,,,;. They are therefore usually used as corrector formulae in predictor— 
corrector schemes. The most common way to do this is to use the (p — 1)th-order 
Adams-—Bashforth formula as predictor, with the pth-order Adams—Moulton formula 
as corrector. This combination can be shown to always produce a scheme of pth order. 
The predictor—corrector scheme we have derived in this section is of this form, with p = 2. 
Of course, for p > 2 the predictor—corrector formula produced is no longer self-starting, 
and other means have to be found to produce the first few values of X. We shall return to 
this topic in the next section. 

It may be noted that one of the alternative methods offered by MATLAB for the 
numerical solution of ODEs is based on the families of Adams—Bashforth and Adams— 
Moulton formulae. 


More accurate solution methods: Runge-Kutta methods 


Another class of higher-order methods comprises the Runge-Kutta methods. The math- 
ematical derivation of these methods is quite complicated and beyond the scope of this book. 
However, their general principle can be explained informally by a graphical argument. 
Figure 2.15 shows a geometrical interpretation of the second-order predictor—corrector 
method introduced in the last section. Starting at the point (¢,, X,,), point A in the diagram, 
the predicted value X,,, is calculated. The line AB has gradient f(t, X,), so the ordinate 
of the point B is the predicted value X,,,. The line AC in the diagram has gradient 
A (trsts X41), the gradient of the direction field of the equation at point B, so point C 
has ordinate X,, + hf(t,,,, X,,;). The midpoint of the line BC, point D, has ordinate X, + 
Le ae Oo aC are X.,,,)), which is the value of X,,, given by the corrector formula. 


x A 
Cc 
D 
Xs ud (Gua; Xs) 7 
Nf (tosis Xu) 
baw TALC Xp) + Abra» Sard 
} Af (ty, X,) 
x, 
oO th, tai t 
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Figure 2.16 

A geometrical 
interpretation of 
the fourth-order 
Runge-Kutta 
method. 


Geometrically speaking, the predictor—corrector scheme can be viewed as the process 
of calculating the gradient of the direction field of the equation at points A and B and 
then assuming that the average gradient of the solution over the interval (t,, ¢,,,) is 
reasonably well estimated by the average of the gradients at these two points. The Euler 
method, of course, is equivalent to assuming that the gradient at point A is a good 
estimate of the average gradient of the solution over the interval (f,, ¢,,,). Given this 
insight, it is unsurprising that the error performance of the predictor—corrector method 
is superior to that of the Euler method. 

Runge-Kutta methods extend this principle by using the gradient at several points in 
the interval (¢,, ¢,,;) to estimate the average gradient of the solution over the interval. 
The most commonly used Runge-Kutta method is a fourth-order one which can be 
expressed as follows: 


C1 = hf ty, Xn) (2.22a) 
cy =hf(t, + 5h, X,+3e) (2.22b) 
C= Af, + >, X, + 5 o>) (2.22¢) 
Cy = hf(t, + h, X, + ¢3) (2.22d) 
eae Gp ee c,) (2.22e) 


Geometrically, this can be understood as the process shown in Figure 2.16. The line AB 
has the same gradient as the equation’s direction field at point A. The ordinate of this 
line at ¢, + th defines point B. The line AC has gradient equal to the direction of the 
direction field at point B. This line defines point C. Finally, a line AD, with gradient 
equal to the direction of the direction field at point C, defines point D. The average 
gradient of the solution over the interval (¢,, ¢,,;) is then estimated from a weighted 
average of the gradients at points A, B, C and D. It is intuitively acceptable that such a 
process is likely to give a highly accurate estimate of the average gradient over the 
interval. 


te 
X, + h(t, +h, X, +65) a 


X, + ft, + $h, X, + $e) 
X, + hf(t, + Fh, X,+ $1) 


Xi, + Mf by Xn) 


C4 


Z(c, + 2c, + 2c; + c4) 


X, 


n 


e) ty tnt t 
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Example 2.10 


Solution 


Figure 2.17 
Computational results 
for Example 2.10. 


As was said before, the mathematical proof that the process defined by (2.22a-e) is 
a fourth-order process is beyond the scope of this text. It is interesting to note that the 
predictor—corrector method defined by (2.21a, b) could also be expressed as 


cy hf th, Ee) 
Ce hf(t, ar h, Xx, ap c\) 


26g SG bce + €) 


This is also of the form of a Runge-Kutta method (the second-order Runge—Kutta 
method), so we find that the second-order Runge-Kutta method and the second- 
order Adams—Bashforth/Adams—Moulton predictor—corrector are, in fact, equivalent 
processes. 


Let X, denote the approximation to the solution of the initial-value problem 


2 
dx _ _x 


dt t+l 


, x(0)=1 


obtained using the fourth-order Runge-Kutta method with a step size h = 0.1, and X, 
that obtained using / = 0.05. Compute the values of X,(4) and X,(¢) for f= 0.1, 0.2,..., 
1.0. Compare these with the values of x(¢), the exact solution of the problem. Compute 
the ratio of the errors in X, and X,. 


The exact solution was given in Example 2.2. The numerical solutions X, and_X, and their 
errors are presented in Figure 2.17. 

This example shows, first, that the Runge—Kutta scheme, being a fourth-order scheme, 
has considerably smaller errors, in absolute terms, than any of the other methods we 
have met so far (note that Figure 2.17 does not give raw errors but errors times 1000!) and, 
second, that the expectation we have that the global error should be O(h’) is roughly 
borne out in practice (the ratio of |x — X,| to |x —_X,| is roughly 16: 1). 


t Xx, Xx, x(t) wowixi®. wax 22% 
|x a X,| 
0.00000  1.0000000 —1.0000000 _1.0000000 
0.10000 1.1053507 11053512 1.1053512 0.00055 0.000 04 0.0682 
0.20000  1.2229733 12229745 —-'1.2229746 0.00133 0.000 09 0.0680 
0.30000 1.3556802 13556825 —1.3556827 0.00246 0.000 17 0.0679 
0.40000 1.5070918 _1.5070957 —_1.5070959 0.00410 0.000 28 0.0678 
0.50000 1.6819805 16819866 1.6819871 0.00653 0.00044 0.0678 
0.60000 1.8867952 18868047 —_1.8868054 0.01020 0.000 69 0.0677 
0.70000  2.1304915  2.1305064 21305074 ~—0.015 92 0.001 08 0.0677 
0.80000  2.4259031  2.4259266  2.4259283 0.02519 0.00171 0.0677 
0.90000  2.7921155 2.7921537 2.7921565 0.04103 0.002 78 0.0677 
1.00000 3.2588214 —-3.2588866 —3.2588914 0.06994 0.004 74 0.0678 
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The table of values in Figure 2.17 can be obtained using MAPLE with the 
appropriate setting of the numerical method. The following worksheet computes 
the solutions specified and composes the required table. 


#set up differential equation 
Olecilsaclvit# (Gee) , ie) ase (ie) 2/7 (eel) same sess (10) =iL 5 
#obtain x1 and x2, the numerical solutions 
xl:=dsolve({deql, init1}, numeric,method=classical [rk4], 
output=listprocedure, stepsize=0.1); 
> x2:=-dsolve({deql, init1},numeric,method=classical[rk4], 
output=listprocedure, stepsize=0.05); 
> #xXa is the analytic solution 
S smecclsolivea (tcc, aiid })) 5 
S jrcimcleyels=0)s 
TEMES ICIe Sa S)5).4 AE IL WE, ILA, ie, OLD. Wie, SIO, Sie, SiO, Sie, 
wil O ae, \in” 


WY NEY 


for i. teem 1 to 10 co 

iEseO, dlrs 

soxiLe =oio (2, sail [2] )) (ie) 2 

38x22 SCI0 (2, x2 || 2] )) (ie) 2 

NOxwels Seyellic (Sulos (eal , Cie (2 , 821) )) )) 

OIMIME TE (IMESIE, t, soll SS<2), seel, alos (Sexil—so<al) “1183 , 

AOS (Sex? —3Sxe))) “LSS, (Sox2 ee) / (eexil —xSxe1)} )) 5 

end do; 


It is interesting to note that the MAPLE results in the right-hand column, the ratio 
of the errors in the two numerical solutions, vary slightly from those in Figure 2.17. 
The results in Figure 2.17 were computed using the high-level programming language 
Pascal which uses a different representation of floating point numbers from that 
used by MAPLE. The variation in the results is an effect of the differing levels of 
precision in the two languages. The differences are, of course, small and do not 
change the overall message obtained from the figure. 


Runge-Kutta schemes are single-step methods in the sense that they only require the 
value of X,,, not the value of X at any steps prior to that. They are therefore entirely self- 
starting, unlike the predictor—corrector and other multistep methods. On the other hand, 
Runge-Kutta methods proceed by effectively creating substeps within each step. There- 
fore they require more evaluations of the function f(t, x) at each step than multistep 
methods of equivalent order of accuracy. For this reason, they are computationally less 
efficient. Because they are self-starting, however, Runge-Kutta methods can be used 
to start the process for multistep methods. An example of an efficient scheme that 
consistently has a fourth-order local error is as follows. Start by taking two steps 
using the fourth-order Runge-Kutta method. At this point values of Xj, X, and _X, are 
available, so, to achieve computational efficiency, change to the three-step fourth- 
order predictor—corrector consisting of the third-order Adams—Bashforth/fourth-order 
Adams—Moulton pair. 
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2.3.9 Exercises 


(Note that Questions 8—15 may be attempted 
using a hand-held calculator, particularly if it 
is of the programmable variety. The arithmetic 
will, however, be found to be tedious, and the 
use of computer assistance is recommended if 
the maximum benefit is to be obtained from 
completing these questions.) 


Using the second-order Adams—Bashforth 
method (start the process with a single step 
using the second-order predictor—corrector 
method), 


(a) compute an estimate of x(0.5) for the initial- 
value problem 
=x’sint—x, x(0)=0.2 
using step size h = 0.1; 
(b) compute an estimate of x(1.2) for the initial- 
value problem 


dx 2 oy 


—=x'e", 


0.5) = 0. 
4 x(0.5) = 0.5 


using step size h = 0.1. 


Using the third-order Adams—Bashforth method 
(start the process with two second-order 
predictor—corrector method steps) compute an 
estimate of x(0.5) for the initial-value problem 


“ = (02 +24, »(0)=1 


using step size h = 0.1. 


Using the second-order predictor—corrector method, 


(a) compute an estimate of x(0.5) for the initial- 
value problem 
dx 


—= = (2t+x)sin2t, 


=0. 
F x(0) 5 


using step size h = 0.05; 
(b) compute an estimate of x(1) for the initial-value 
problem 


dx l+x 


a me 


using step size h = 0.1. 
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12 


is 


Write down the first three terms of the Taylor series 
expansions of the functions 


dx 
—(t—2h 
rr ( ) 
about x(t). Use these two equations to eliminate 
2 3 
Gx) and 1X1) 
dt dt 


from the Taylor series expansion of the function 
x(t +h) about x(t). Show that the resulting formula 
for x(t + A) is the third member of the Adams— 
Bashforth family, and hence confirm that this 
Adams-Bashforth method is a third-order method. 


Write down the first three terms of the Taylor series 
expansions of the functions 


about x(t). Use these two equations to eliminate 
2 3 
fxr) and 12H 
dt dt 


from the Taylor series expansion of the function 
x(t + h) about x(t). Show that the resulting formula 
for x(t + A) is the third member of the Adams— 
Moulton family, and hence confirm that this 
Adams—Moulton method is a third-order method. 


Write down the first four terms of the Taylor series 
expansion of the function x(t — 4) about x(t), and the 
first three terms of the expansion of the function 


dx 
—(t—-h 
dy (t—h) 
about x(t). Use these two equations to eliminate 
2 3 
Gx) and 2X1) 
dt dt 


from the Taylor series expansion of the function 
x(t + h) about x(t). Show that the resulting formula is 


Xia = 4X, + 5X,1 + A(4F, + 2F,,1) + O(A*) 
Show that this method is a linear combination of the 
second-order Adams—Bashforth method and the 
central difference method (that is, the scheme based 


on (2.9)). What do you think, in view of this, might 
be its disadvantages? 
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Using the third-order Adams—Bashforth—Moulton 
predictor—corrector method (that is, the second- 
order Adams—Bashforth formula as predictor and 
the third-order Adams—Moulton formula as 
corrector), compute an estimate of x(0.5) for 
the initial-value problem 

dx 


S=x +2’, 


(0.3) = 0.1 
a x(0.3) =0 


using step size h = 0.05. (You will need to employ 
another method for the first step to start this scheme 
—use the fourth-order Runge-Kutta method). 


Using the fourth-order Runge-Kutta method, 


(a) compute an estimate of x(0.75) for the initial- 
value problem 


Ot ea xt, x(0) = 1 
dt 


using step size h = 0.15; 
(b) compute an estimate of x(2) for the initial-value 


problem 
d 1 
a aie x(1) =2 


using step size h=0.1. 


Consider the initial-value problem 


dk 32472, 3(0)=-1 

dt 

(a) Compute estimates of x(2) using the second- 
order Adams—Bashforth scheme (using the 
second-order predictor—corrector to start the 
computation) with step sizes h = 0.2 and 0.1. 
From these two estimates of x(2) estimate what 
step size would be needed to compute an 
estimate of x(2) accurate to 3dp. Compute X(2), 
first using your estimated step size and second 
using half your estimated step size. Does the 
required accuracy appear to have been achieved? 

(b) Compute estimates of x(2) using the second- 
order predictor—corrector scheme with step 
sizes h = 0.2 and 0.1. From these two estimates 


of x(2) estimate what step size would be 
needed with this scheme to compute an 
estimate of x(2) accurate to 3dp. Compute 
X(2), first using your estimated step size and 
second using half your estimated step size. 
Does the required accuracy appear to have 
been achieved? 

(c) Compute estimates of x(2) using the fourth- 
order Runge—Kutta scheme with step sizes 
h = 0.4 and 0.2. From these two estimates of 
x(2) estimate what step size would be needed to 
compute an estimate of x(2) accurate to 5 dp. 
Compute X(3), first using your estimated step 
size and second using half your estimated step 
size. Does the required accuracy appear to have 
been achieved? 


For the initial-value problem 


dx = 
OX _ 2 et 


I=1 
di » x(1) 


find, by any method, an estimate, accurate to 5dp, of 
the value of x(3). 


Note: All of the exercises in this section can be 
completed by programming the algorithms in a 
high-level computer language such as Pascal, 

C and Java. Programming in a similar high-level 
style can be achieved using the language constructs 
embedded within the MATLAB and MAPLE 
packages. MAPLE, as we have already seen, 
and MATLAB also allow a higher-level style 

of programming using their built-in procedures 
for numerical solution of ODEs. Both MATLAB 
and MAPLE have very sophisticated built-in 
procedures, but MAPLE also allows the user 

to specify that it should use simpler algorithms 
(which it calls ‘classic’ algorithms). Amongst 
these simpler algorithms are many of the 
algorithms we discuss in this chapter. In the 
preceding exercise set, those which specify the 
Runge-Kutta method and the second-order 
predictor—corrector could be completed using 
MAPLE’s dsolve procedure specifying the 
relevant ‘classic’ solution methods. 
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2.3.10 


Example 2.11 


Figure 2.18 

The analytical 
solutions of 
(2.23) and (2.24). 


Stiff equations 


There is a class of differential equations, known as stiff differential equations, that are 
apt to be somewhat troublesome to solve numerically. It is beyond the scope of this text 
to explore the topic of stiff equations in any great detail. It is, however, important to be 
aware of the possibility of difficulties from this source and to be able to recognize the 
sort of equations that are likely to be stiff. In that spirit we shall present a very informal 
treatment of stiff equations and the sort of troubles that they cause. Example 2.11 shows 
the sort of behaviour that is typical of stiff differential equations. 


The equation 


rF =l-x, x(0)=2 (2.23) 
has analytical solution x = 1 + e’. The equation 

dx = 

a 5001 —x)+50e%, x(0)=2 (2.24) 


has analytical solution x = 1 + 4 (50 e’— e°”). The two solutions are shown in 
Figure 2.18. 

Suppose that it were not possible to solve the two equations analytically and 
that numerical solutions must be sought. The form of the two solutions shown in 
Figure 2.18 is not very different, and it might be supposed (at least naively) that the 
numerical solution of the two equations would present similar problems. This, however, 
is far from the case. 

Figure 2.19 shows the results of solving the two equations using the second-order 
predictor—corrector method with step size h = 0.01. The numerical and exact solutions 
of (2.23) are denoted by X, and x, respectively, and those of (2.24) by X, and x,. The 
third and fifth columns give the errors in the numerical solutions (compared with the 
exact solutions), and the last column gives the ratio of the errors. The solution X, is seen 
to be considerably more accurate than X,, using the same step size. 
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Figure 2.19 
Computational results 
for Example 2.11; 
h=0.01. 


Figure 2.20 
Computational results 
for Example 2.11; 
h=0.025. 


Figure 2.21 
Computational results 
for Example 2.11; 
h=0.05. 


t xX, IX. - Xal X, |X, — xy] Ratio of 


errors 
0.000 00 2.000 00 0.000 000 2.000 00 0.000 000 
0.100 00 1.904 84 0.000 002 1.923 15 0.000 017 11.264 68 
0.200 00 1.818 73 0.000 003 1.835 47 0.000 028 10.022 19 
0.300 00 1.740 82 0.000 004 1.755 96 0.000 026 6.864 34 
0.400 00 1.670 32 0.000 005 1.684 02 0.000 023 5.15007 
0.500 00 1.606 54 0.000 005 1.618 93 0.000 021 4.12006 
0.600 00 1.548 82 0.000 006 1.560 03 0.000 019 3.433 38 
0.700 00 1.496 59 0.000 006 1.506 74 0.000 017 2.942 90 
0.800 00 1.449 34 0.000 006 1.458 51 0.000 016 2.575 03 
0.900 00 1.406 58 0.000 006 1.41488 0.000 014 2.288 92 
1.000 00 1.367 89 0.000 006 1.375 40 0.000 013 2.060 02 
t X, |X, - xal X, |X, — xl Ratio of 
errors 
0.000 00 2.000 00 0.000 000 2.000 00 0.000 000 
0.10000 1.904 85 0.000 010 1.922 04 0.001 123 116.951 24 
0.200 00 1.81875 0.000 017 1.835 67 0.000 231 13.270 10 
0.300 00 1.740 84 0.000 024 1.756 25 0.000 317 13.438 84 
0.400 00 1.67035 0.000 028 1.684 30 0.000 296 10.384 39 
0.500 00 1.606 56 0.000 032 1.61918 0.000 268 8.328 98 
0.600 00 1.548 85 0.000 035 1.560 25 0.000 243 6.942 36 
0.700 00 1.496 62 0.000 037 1.506 94 0.000 220 5.950 68 
0.800 00 1.449 37 0.000 038 1.458 70 0.000 199 5.206 82 
0.900 00 1.40661 0.000 039 1.41505 0.000 180 4.628 26 
1.000 00 1.367 92 0.000 039 1.375 55 0.000 163 4.165 42 
t x, IX. - Xl X, |X, — Xp 
0.000 00 2.000 00 0.000 000 2.000 00 0.000 000 
0.100 00 1.904 88 0.000 039 1.873 43 0.049 740 
0.200 00 1.818 80 0.000 071 1.707 36 0.128 075 
0.300 00 1.74091 0.000 096 1.42102 0.334914 
0.400 00 1.670 44 0.000 116 0.802 59 0.881 408 
0.500 00 1.606 66 0.000 131 —0.705 87 2.324778 
0.600 00 1.548 95 0.000 142 —4.576 42 6.136 434 
0.700 00 1.496 74 0.000 150 —14.695 10 16.201 818 
0.800 00 1.449 48 0.000 156 —41.322 43 42.780 932 
0.900 00 1.406 73 0.000 158 —111.55173 112.966 595 
1.000 00 1.368 04 0.000 159 —296.925 40 298.300 783 


Figure 2.20 is similar to Figure 2.19, but with a step size h = 0.025. As we might 
expect, the error in the solution _X, is larger by a factor of roughly six (the global error 
of the second-order predictor—corrector method is O(h’)). The errors in X,, however, 
are larger by more than the expected factor, as is evidenced by the increase in the ratio 
of the error in_X,, to that in_X,. 

Figure 2.21 shows the results obtained using a step size h = 0.05. The errors in_X, 
are again larger by about the factor expected (25 when compared with Figure 2.19). The 


www.2Ofile.org 


2.3 NUMERICAL SOLUTION OF FIRST-ORDER ORDINARY DIFFERENTIAL EQUATIONS 149 


2.3.11 


solution X,, however, shows little relationship to the exact solution x, — so little that the 
error at ¢= | is over 20 000% of the exact solution. Obviously a numerical method that 
causes such large errors to accumulate is not at all satisfactory. 

In Section 2.3.5 we met the idea that some numerical methods can, when applied to 
some classes of differential equation, show instability. What has happened here is, of 
course, that the predictor—corrector method is showing instability when used to solve 
(2.24) with a step size larger than some critical limit. Unfortunately the same behaviour 
is also manifest by the other methods that we have already come across — the problem 
lies with the equation (2.24), which is an example of a stiff differential equation. 


The typical pattern with stiff differential equations is that, in order to avoid instabil- 
ity, the step size used to solve the equation using normal numerical methods must be 
very small when compared with the interval over which the equation is to be solved. In 
other words, the number of steps to be taken is very large and the solution is costly in 
time and computing resources. Essentially, stiff equations are equations whose solution 
contains terms involving widely varying time scales. That (2.24) is of this type is evid- 
enced by the presence of terms in both e‘ and e~” in the analytical solution. In order 
to solve such equations accurately, a step must be chosen that is small enough to cope 
with the shortest time scale. If the solution is required for times comparable to the long 
time scales, this can mean that very large numbers of steps are needed and the computer 
processing time needed to solve the problem becomes prohibitive. In Example 2.11 the 
time scale of the rapidly varying and the more slowly varying components of the solu- 
tion differed by only a factor of 50. It is not unusual, in the physical problems arising 
from engineering investigations, to find time scales differing by three or more orders 
of magnitude; that is, factors of 1000 or more. In these cases the problems caused 
are proportionately amplified. Fortunately a number of numerical methods that are 
particularly efficient at solving stiff differential equations have been developed. It is 
beyond the scope of this text to treat these in any detail. 

From the engineering point of view, the implication of the existence of stiff equations 
is that engineers must be aware of the possibility of meeting such equations and also of the 
nature of the difficulty for the numerical methods — the widely varying time scales inherent 
in the problem. It is probably easier to recognize that an engineering problem is likely to 
give rise to a stiff equation or equations because of the physical nature of the problem than 
it is to recognize a stiff equation in its abstract form isolated from the engineering con- 
text from which it arose. As is often the case, a judicious combination of mathematical 
reasoning and engineering intuition is more powerful than either approach in isolation. 

Both MAPLE and MATLAB feature procedures for the numerical solution of ODEs 
which are designed to deal efficiently with stiff equations. The user may be tempted to 
think that a simple way to negotiate the problem of stiff equations is to use the stiff equation 
solvers for all ordinary differential equations. However, the stiff equation methods are less 
computationally efficient for non-stiff equations so it is worth trying to identify which 
type of equation one is facing and using the most appropriate methods. 


Computer software libraries and the ‘state of the art’ 


In the last few sections we have built up some basic methods for the integration of first- 
order ordinary differential equations. These methods, particularly the more sophisticated 
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ones — the fourth-order Runge-Kutta and the predictor—corrector methods — suffice for 
many of the problems arising in engineering practice. However, for more demanding 
problems — demanding in terms of the scale of the problem or because the problem is 
characterized by ill behaviour of some form — there exist more sophisticated methods 
than those we are able to present in this book. 

All the methods that we have presented in the last few sections use a fixed step size. 
Among the more sophisticated methods to which we have just alluded are some that use 
a variable step size. In Section 2.3.6 we showed how Richardson extrapolation can be 
used to estimate the size of the error in a numerical solution and, furthermore, to estim- 
ate the step size that should be used in order to compute a solution of a differential 
equation to some desired accuracy. The principle of the variable-step methods is that a 
running check is kept of the estimated error in the solution being computed. The error 
may be estimated by a formula derived along principles similar to that of Richardson 
extrapolation. This running estimate of the error is used to predict, at any point in the 
computation, how large a step can be taken while still computing a solution within any 
given error bound specified by the user. The step size used in the solution can be altered 
accordingly. If the error is approaching the limits of what is acceptable then the step 
size can be reduced; if it is very much smaller than that which can be tolerated then the step 
size may be increased in the interests of speedy and efficient computing. For multistep 
methods the change of step size can lead to quite complicated formulae or procedures. 
As an alternative, or in addition, to a change of step size, changes can be made in the 
order of the integration formula used. When increased accuracy is required, instead 
of reducing the step size, the order of the integration method can be increased, and 
vice versa. Implementations of the best of these more sophisticated schemes are readily 
available in software packages, such as MAPLE and MATLAB, and software libraries 
such as the NAG library. 

The availability of complex and sophisticated ‘state of the art’ methods is not the 
only argument for the use of software packages and libraries. It is a good engineering 
principle that, if an engineer wishes to design and construct a reliable engineering artefact, 
tried and proven components of known reliability and performance characteristics 
should be used. This principle can also be extended to engineering software. It is almost 
always both more efficient, in terms of expenditure of time and intellectual energy, and 
more reliable, in terms of elimination of bugs and unwanted side-effects, to use soft- 
ware from a known and proven source than to write programs from scratch. 

For both of the foregoing reasons, when reliable mathematical packages, such as 
MAPLE and MATLAB, and software libraries are available, their use is strongly 
recommended. MAPLE is arguably the leading mathematical software package available 
today, offering both symbolic manipulation (computer algebra) and numerical problem 
solving across the whole span of mathematics. Amongst these, as we have already 
noted, MAPLE includes routines for the numerical solution of systems of ordinary 
differential equations. These routines are highly sophisticated, offering alternative 
methods suitable for stiff and non-stiff problems, using fixed time steps or variable time 
steps and optimized either for speed or for accuracy. The MATLAB package, with its 
Simulink continuous system modelling add-on, also offers sophisticated facilities for 
solving differential equations numerically. Again the package offers the choice of 
both fixed and variable time step methods, methods suitable for stiff problems as well 
as non-stiff ones, and a choice of optimizations aimed at either best speed or highest 
accuracy. Amongst the best known, and probably the most widely used, library of 
software procedures today is the NAG library. This library has a long history and has 
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been compiled by leading experts in the field of numerical mathematics. Routines are 
available in a variety of programming languages. The routines provided for the solution 
of ordinary differential equations again encompass a variety of methods chosen to deal 
with stiff and non-stiff problems and to offer the user considerable flexibility in choice 
of method to suit every possible engineering requirement. By choosing an appropriate, 
high-quality software package or library the engineer can be assured that the imple- 
mentation will be, as far as possible, bug free, that the methods used will be efficient 
and reliable, and that the algorithms will have been chosen from the best ‘state of the 
art’ methods. 

It is tempting to believe that the use of software libraries solves all the problems of 
numerical analysis that an engineering user is likely to meet. Faced with a problem for 
which analytical methods fail, the engineer simply needs to thumb through the index to 
some numerical analysis software library until a method for solving the type of problem 
currently faced is found. Unfortunately such undiscerning use of packaged software 
will almost certainly, sooner or later, lead to a gross failure of some sort. If the user is 
fortunate, the software will be sophisticated enough to detect that the problem posed is 
outside its capabilities and to return an error message to that effect. If the user is less 
fortunate, the writer of the software will not have been able to foresee all the possible 
uses and misuses to which the software might be subjected and the software will not be 
proof against such use outside its range of applicability. In that case the software may 
produce seemingly valid answers while giving no indication of any potential problem. 
Under such circumstances the undiscerning user of engineering software is on the verge 
of committing a major engineering blunder. From such circumstances result failed 
bridges and crashed aircraft! It has been the objective of these sections on the numerical 
solution of differential equations both to equip readers with numerical methods suitable 
for the less demanding problems that will arise in their engineering careers and to give 
them sufficient understanding of the basics of this branch of numerical analysis that 
they may become discriminating, intelligent and wary users of packaged software and 
other aids to numerical computing. 


Numerical solution of second- and higher-order 
differential equations 

Obviously, the classes of second- and higher-order differential equations that can be 
solved analytically, while representing an important subset of the totality of such 
equations, are relatively restricted. Just as for first-order equations, those for which no 
analytical solution exists can still be solved by numerical means. The numerical solu- 


tion of second- and higher-order equations does not, in fact, need any significant new 
mathematical theory or technique. 


Numerical solution of coupled first-order equations 


In Section 2.3 we met various methods for the numerical solution of equations of the 
form 


& = ,x) 


www.2Ofile.org 


152 NUMERICAL SOLUTION OF ORDINARY DIFFERENTIAL EQUATIONS 


that is, first-order differential equations involving a single dependent variable and a 
single independent variable. However it is possible to have sets of coupled first-order 
equations, each involving the same independent variable but with more than one 
dependent variable. An example of these types of equation is 


a =4- 5 2x (2.25) 
2 =2x +xy-t (2.25b) 


This is a pair of differential equations in the dependent variables x and y with the inde- 
pendent variable t. The derivative of each of the dependent variables depends not only 
on itself and on the independent variable ¢, but also on the other dependent variable. 
Neither of the equations can be solved in isolation or independently of the other — both 
must be solved simultaneously, or side by side. A pair of coupled differential equations 
such as (2.25) may be characterized as 


a =f, 69) (2.26a) 
2 =£(E% (2.26b) 


For a set of p such equations it is convenient to denote the dependent variables not by 
X, y,Z,... but by x), x), x3,..., x, and the set of equations by 


dx; ; 
— = fi(t, X1, X2,... 5 Xp) (@=1,2,...,p) 
dt 
or equivalently, using vector notation, 
d 
— [x] =f(t, x 
© [x] =f.) 


where x(f) is a vector function of t given by 


x“H=[) x(t) ... x,(OT" 


S(t, x) is a vector-valued function of the scalar variable ¢ and the vector variable x. 
The Euler method for the solution of a single differential equation takes the 
form 
Xntt a X, ap hf (th X,) 
If we were to try to apply this method to (2.26a), we should obtain 


ACs = 2G ap f(t Xe Ye) 


In other words, the value of X,,,, depends not only on ¢, and X,, but also on Y,,. In the same 
way, we would obtain 


Me 


n+l 


= Y, + hf Xn Yn) 
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Solution 


for Y,,,. In practice, this means that to solve two simultaneous differential equations, 
we must advance the solution of both equations simultaneously in the manner shown in 


Example 2.12. 


Find the value of X(1.4) satisfying the following initial-value problem: 


& =x-y'+xt, x(1)=0.5 
“ =2x?+xy-t, y)=12 


using the Euler method with time step A = 0.1. 


The right-hand sides of the two equations will be denoted by f((¢, x, y) and A(t, x, y) 
respectively, so 


fit, x, y)=x—-y? +xt and Silt, x, y) = 2x* +xy-t 


The initial condition is imposed at f= 1, so ¢, will denote 1 + nh, X,, will denote X(1 + nh), 
and Y, will denote Y(1 + nh). Then we have 


X, =X + Afi(to, Xo Vo) Y, =¥o + Afi(to, Xo, Vo) 
=0.5+ 0.171, 0.5, 1.2) = 1.2 + 0.14, 0.5, 1.2) 
= 0.4560 = 1.2100 


for the first step. The next step is therefore 


X= X, + hfilt, X,, Y) Y,=Y, + Aft, X,Y) 
= 0.4560 = 1.2100 
+ 0.1f,(1.1, 0.4560, 1.2100) + 0.1f,(1.1, 0.4560, 1.2100) 
= 0.4054 = 1.1968 


and the third step is 


X, = 0.4054 Y; = 1.1968 
+ 0.1f(1.2, 0.4054, 1.1968) + 0.1f(1.2, 0.4054, 1.1968) 
= 0.3513 = 1.1581 


Finally, we obtain 
X, = 0.3513 + 0.17,1.3, 0.3513, 1.1581) 
= 0.2980 


Hence we have X(1.4) = 0.2980. 
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2) MAPLE’s dsolve procedure can find the numerical solution of sets of coupled 
ordinary differential equations as readily as for a single differential equation. The 
following worksheet finds the solution required in the example above. 


> #set up the two differential equations 
Ss cleo sachuiriz (s< (ie) pie) ee (ie) > (alse) a (Ge) 2 
CAG ¢=CHLIEE (7 (ie) pe) Seer (ie) 2 tec (ie) Av (ie) es 
deqsystem:=deq1,deq2; 
> #set up the initial conditions 
= iaateesk (1) =0.5,w(ljel.2e 
> #procedure “dsolve” used to solve s system of two coupled 
differential equations 
> sol:=dsolve({deqsystem, inits}, numeric, 
method=classical[foreuler] , output=listprocedure, 
stepsize=0.1); 


> #obtain numerical solution required 
S saeco (2 , SOL 12] )) esse (IL. 4) 2 


The principle of solving the two equations side by side extends in exactly the same 
way to the solution of more than two simultaneous equations and to the solution of 
simultaneous differential equations by methods other than the Euler method. 


Example 2.13 Find the value of X(1.4) satisfying the following initial-value problem: 


“ =x-y?+xt, x(1)=0.5 
WY oP eye y= 12 
dt 


using the second-order predictor—corrector method with time step 4 = 0.1. 


Solution _ First step: 


predictor 
X, = Xo + (tos Xo» Vo) Y, = + Hef\(tos Xo» Vo) 
= 0.4560 = 1.2100 
corrector 
X, =X t+ SALA(to, Xo, Yo) Y,=y+ SAL A(t, Xo, Vo) 
+ fit, X, YI + f(t, X,Y) 
= 0.5 + 0.05[/C, 0.5, 1.2) =1.2+4+0.05[ Ad, 0.5, 1.2) 
+f,C1.1, 0.456, 1.21)] + f,(1.1, 0.456, 1.21)] 
= 0.4527 = 1.1984 
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Second step: 
predictor 
X, =X, + hfit,, X, Y)) 
= 0.4042 
corrector 
X, =X + FLA, X, Y) 
+ filb, %, ¥,)] 
= 0.4527 
+ 0.05[f,(1.1, 0.4527, 1.1984) 
+f,(1.2, 0.4042, 1.1836)] 
= 0.4028 


Third step: 


predictor 
X; =X) + hilt, X, Y>) 
= 0.3542 
corrector 
X, =X, + 5 hl fib, X, Y,) 
+ hilt, ¥, Y3)1 
= 0.4028 
+ 0.05, /,(1.2, 0.4028, 1.1713) 
+ fi(1.3, 0.3542, 1.1309)] 
= 0.3553 


Fourth step: 


predictor 
X= Xy+ hf\(t, X3, Ys) 
= 0.3119 


corrector 


X, = X34 SALA Xs, Ys) + flt Xs, Y,)I 
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y= Y + f(t), X1, Y,) 
= 1.1836 


Y=Y,+ SALA, Xi, Y,) 
+ fil, %, ¥,)] 
= 1.1984 
+ 0.05[ f(1.1, 0.4527, 1.1984) 
+ f,(1.2, 0.4042, 1.1836)] 
= 1.1713 


Y,=¥%+ hfr(t, X>, Y) 
= 1.1309 


Y,=¥,+ LAL Alt, Xy Y,) 
+ fil, X,Y) 
= 1.1713 
+ 0.05[ (1.2, 0.4028, 1.1713) 
+ f(1.3, 0.3542, 1.1309)] 
= 1.1186 


Y= %,+ hfr(ts, X3, Y3) 
= 1.0536 


= 0.3553 + 0.05[ f,(1.3, 0.3553, 1.1186) + f,(1.4, 0.3119, 1.0536)] 


Hence finally we have X(1.4) = 0.3155. 
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2.4.2 


Example 2.14 


The MAPLE worksheet at the end of Example 2.12 can be easily modified to repro- 
duce the solution of Example 2.13 by changing the name of the required numerical 
method from foreuler to heunform. 


It should be obvious from Example 2.13 that the main drawback of extending 
the methods we already have at our disposal to sets of differential equations is the 
additional labour and tedium of the computations. Intrinsically, the computations are 
no more difficult, merely much more laborious — a prime example of a problem ripe 
for computerization. 


State-space representation of higher-order systems 


The solution of differential equation initial-value problems of order greater than one can 
be reduced to the solution of a set of first-order differential equations using the state-space 
representation introduced in Section 1.9. This is achieved by a simple transformation, 
illustrated by Example 2.14. 


The initial-value problem 


2 

G2 x St -xf=l 
dt dt 

can be transformed into two coupled first-order differential equations by introducing 

an additional variable 


2 


, x(0)=12, %(0)=0.8 
dt 


_ dx 
dt 
With this definition, we have 
d'x _ dy 
de dt 
and so the differential equation becomes 


dy 
qt t= st 
Thus the original differential equation can be replaced by a pair of coupled first-order 


differential equations, together with initial conditions: 


2 


dx 

—=y, 0)= 1.2 

ae? x(0) 

dy_ a 2,17 = 
qe a y(0) = 0.8 


This process can be extended to transform a pth-order initial-value problem into a 
set of p first-order equations, each with an initial condition. Once the original equation 
has been transformed in this way, its solution by numerical methods is just the same 
as if it had been a set of coupled equations in the first place. 
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Solution 


157 


Find the value of X(0.2) satisfying the initial-value problem 
3 2 2 
dey $84 Py = 0, x00) =1, Bey=05, (0) =-02 
dt dt dt dt dt 


using the fourth-order Runge-Kutta scheme with step size h = 0.05. 


Since this is a third-order equation, we need to introduce two new variables: 


2 

y= dx and z= 2= ds 

dt dt dr 

Then the equation is transformed into a set of three first-order differential equations 

dx 
oe x(0) 
dy 
== 0) = 0.5 
ae y(0) 
dz _ 2 aa 
—==-xtz—-ty+t'x z(0) =—0.2 


dt 


Applied to the set of differential equations 


“ =filt,x,¥.2) 
2 = filt, x,y, 2) 


& =f,(t, x, y, 2) 


the Runge-Kutta scheme is of the form 


11 = Aft, Xn Yas Zn) 
Co = Mfg ty, Xns Yn Zn) 
C31 = Aft Xn Yn Zn) 
Cy = hf, + th, X, + Seis ae ne Z, + $C31) 
Cy = Aft, + th, X,+ sCis Y78 eas Z, + sai) 
Ca = Afy(t, + th, X, + Sis oe ore Z, + $e) 
C3 = hf, + sh, X, + stig ne 5 Cop, Z, + + C39) 
Cy; = Aft, + sh, A, + 5 Clas Yet 5 C25 Z, + 5 C39) 
C33 = Afy(t, + sh, X,+ 5 Cia: Y,+ 5 Cops Z, + io) 
C14 = Aft, +h, X, + Cys, Y, + Cp, Z, + C33) 
Coq = Aft, +h, X, + C13, Y, + Cn, Z, + C33) 


Cx = Aft, +h, X, + C13, Y,, + Crs, Z, + C33) 
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Xi =X, + t(cy + 2¢)) + 213 + Cy4) 
Yin = Y, + t (Co + 205) + 293 + Co4) 
Ziv = Z, + 1 (C31 + 203) + 2033 + C34) 
Note that each of the four substeps of the Runge-Kutta scheme must be carried out in 
parallel on each of the equations, since the intermediate values for all the independent 
variables are needed in the next substep for each variable; for instance, the computation 


of c,; requires not only the value of c,, but also the values of c,, and c3). The first step of 
the computation in this case proceeds thus: 


Xj =X = 1 Y=M = 0.5 Zo = 2% =—0.2 
C11 = Af(to» Xov Yor Zo) 
=hY, 
= 0.025 000 Cn = Malt, Xo. Yo, Zo) 
=hZ, 
=—0.010000 C31 = hfx(to, Xo, Yo, Zo) 
= h(-XotpZy — ty Yo + tX0) 
= 0.000 000 
Ci = Aifilty + 4h, Xp + bey, Yo + bey, Zy + 431) 
=h(Y,+ $Co1) 
= 0.024 750 
Cy = Aft) + th, Xyt+ 5 Chis Yy+ $n, Zt $031) 
=h(Z)+ ses) 
= -0.010 000 


Cay = Mfg + Fh, Xo + FCs Yo + $C, Zo + $031) 
= h(-(Xq + Fer )(to + $A)(Zo + $¢31) 
— (ty + $A)(Yo + $en1) + (to + SAY Xo + Fen) 
= —0.000 334 
C13 = Af\(ty + ait, X)+ 5 C125 Yyt+ 5 Cons Zo to) 
= h(¥) + be) 
= 0.024 750 
Co3 = Af(ty + th, Xy+ S C10, y+ $ Cop, Zt + C39) 
=h(Z)+ } C9) 
=—0.010008 
C33 = Afy(ty + sh, Xo+ 5G: Yot+ on Zot 5x) 
= h(-(X%) + 4 eia\(to + Sh)(Zy + 5039) 
— (ty + SAY + Cx) + ly + SAV X + § 12) 
= —0.000 334 
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Cia = Nf (ty + h, Xo + C13, Yo + Cr3, Zp + C33) 
=h(Y% + c93) 
= 0.024 499 
Cog = Nifty + h, Xo + C13, Yo + x3, Zp + C33) 
= A(Zy + C33) 
=-—0.010016 
Caq = hfy(ty + h, Xp + C13, Yo + Co3, Zq + C33) 
= h(-(X + €13)(ty + AY(Zo + C33) 
— (ty + A)(Ly + Co3) + (ty + AY(XK + C3) 
= —0.000 584 
X, = 1.024750, Y, = 0.489 994, Z, =—0.200 320 


The second and subsequent steps are similar — we shall not present the details of the 
computations. It should be obvious by now that computations like these are sufficiently 
tedious to justify the effort of writing a computer program to carry out the actual arith- 
metic. The essential point for the reader to grasp is not the mechanics, but rather the 
principle whereby methods for the solution of first-order differential equations can be 
extended to the solution of sets of equations and hence to higher-order equations. 


2) Again MAPLE could be used to find the numerical solution of this set of coupled 
ordinary differential equations. However, the MAPLE dsolve procedure is also able 
to do the conversion of the higher-order equation into a set of first-order equations 
internally so the numerical solution of the example above using the fourth-order 
Runge-Kutta algorithm could be achieved with the following worksheet. 


> #set up the differential equation 
S Cleo selaliew (se (ie) , ie, te, te) ee (ie) ie sCliiese (xi(te) , i, ie) 
are GlaLinie (Se (10) pte) te Bose (ic) Op 
> #set up the initial conditions 
S dinvceses (0) =i ,iD(s) (0) =0.5,D(OG:s) ) (O) S025 
> #procedure “dsolve” used to solve third order 
differential equations 
> sol:=dsolve({deq, inits}, numeric,method=classical[rk4], 
output=listprocedure, stepsize=0.05); 
> #obtain the numerical solution required 
SS seen Soo(2, SO 121) ) peee(@ 10'S) psex(O., 2) 2 
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18 


2.4.3 Exercises 


Transform the following initial-value problems 
into sets of first-order differential equations with 


appropriate initial conditions: 


2 
(a) dx 
dt dt 


= dX yy _ 
x(0)=1, FO)=2 


dx 


(b) +4@?-1)'?=0 
dt 


= dx (4) _ 
I)=2, ST) =0.5 


(c) d°x (“ 


— - sin ft) + 4x=0 
dt dt 


ee 
x(0)=0, FO) =0 


3 2: 
(4) $24 14 60 B= 0% 
de de dt 
x0)=1, @=2, 4@=o0 
dt dt 
=} 2 
(e) day PX 4? = sing 
dt dt 
x=1, &ay=0, &a-= 
dt dt 


3 1/2 2 
(f) (s) +1184"? =0 
dt dt 


. 2 
x2)=0, %y=0, a=2 
dt dt 


eGGe A ae e 


x(0) =0, 


4 2 
(g) dx, de, P= Int, 
dt dt 
2 3 
dx@=4, $2 @=-3 
dt dt 
(ny $24 (#- 1) fe ee 
dt dt dt t 
+4t—5 
x0)=a, 2@=0, =, 
dt dt 


dx (0) =0, 
dt 

3 
dx) =0 
dt 


20 


Zl 


22 


E} 


Find the value of X(0.3) for the initial-value 
problem 
2 
dx, PO. y= sins, 0) =0, 2=1 
dt dt dt 


using the Euler method with step size h = 0.1. 


The second-order Adams—Bashforth method for 
the integration of a single first-order differential 
equation 


dx 
—=f(t. 
dy f(t, x) 
is 
Xr =X t+ ALBA ty Xn) — Mbps Xn) 


Write down the appropriate equations for applying 
the same method to the solution of the pair of 
differential equations 


dx _ dy _ 
dt =fi(t x; y), dt = f/xlt, x, y) 


Hence find the value of X(0.3) for the initial-value 
problem 
2 
dex, 2d. sing, x(0)=0, 


: dx) =1 
dt dt dt 


using this Adams—Bashforth method with step size 
h =0.1. Use the second-order predictor—corrector 
method for the first step to start the computation. 


Use the second-order predictor—corrector 

method (that is, the first-order Adams—Bashforth 
formula as predictor and the second-order 
Adams—Moulton formula as corrector) to compute 
an approximation X(0.65) to the solution x(0.65) 
of the initial-value problem 


3 2 2 
£4 ~~ yte+(8) -x =0 
dt dt dt 

dx 


x0.5)=-1, “(s)=1, 4205)=2 
dt dt 


using a step size h = 0.05. 


Write a computer program to solve the initial-value 
problem 


2: 
dx, 2a. >= sint, x(0)=0, 


: dx Q) =1 
dt dt dt 
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using the fourth-order Runge-Kutta method. Use 
your program to find the value of X(1.6) using 
step sizes h = 0.4 and 0.2. Estimate the accuracy 
of your value of X(1.6) and estimate the step size 
that would be necessary to obtain a value of X(1.6) 
accurate to 6dp. 


Write a computer program to solve the initial- 
value problem 


3 2 2 
awe, 23+ (S) x =0 
dt dt dt 


: dx = d°x = 
x(0.5)=-1, =(05)=1, —>(0.5)=2 
dt dt 


2.4.4 


Boundary-value problems 


using the third-order predictor—corrector method 
(that is, the second-order Adams—Bashforth 
formula as predictor with the third-order Adams— 
Moulton as corrector). Use the fourth-order Runge— 
Kutta method to overcome the starting problem 
with this process. Use your program to find the 
value of X(2.2) using step sizes h = 0.1 and 0.05. 
Estimate the accuracy of your value of X(2.2) and 
estimate the step size that would be necessary to 
obtain a value of X(2.2) accurate to 6dp. 


Note: The comment on the use of high-level computer 
language and the MATLAB and MAPLE packages 
at the end of Section 2.3.9 is equally applicable to 

the immediately preceding exercises in this section. 


Because first-order ordinary differential equations only have one boundary condition, 
that condition can always be treated as an initial condition. Once we turn to second- and 
higher-order differential equations, there are, at least for fully determined problems, two 
or more boundary conditions. If the boundary conditions are all imposed at the same point 
then the problem is an initial-value problem and can be solved by the methods we have 
already described. The problems that have been used as illustrations in Sections 2.4.1 
and 2.4.2 were all initial-value problems. Boundary-value problems are somewhat more 
difficult to solve than initial-value problems. 

To illustrate the difficulties of boundary-value problems, let us consider second-order 
differential equations. These have two boundary conditions. If they are both imposed at the 
same point (and so are initial conditions), the conditions will usually be a value of the 
dependent variable and of its derivative, for instance a problem like 


LIQ] =f), x(a) =p, “(a)=4 


where L is some differential operator. Occasionally, a mixed boundary condition such as 


dK pe 
Cx(a) + ae (a)=p 


will arise. Provided that a second boundary condition on x or dx/dt is imposed at the same 
point, this causes no difficulty, since the boundary conditions can be decoupled, that is 
solved to give values of x(a) and (dx/df)(a), before the problem is solved. 

If the two boundary conditions are imposed at different points then they could con- 
sist of two values of the dependent variable, the value of the dependent variable at 
one boundary and its derivative at the other, or even linear combinations of the values 
of the dependent variable and its derivative. For instance, we may have 


LI] =f, 


or 


x(a) = p, 


x(b) = q 


LIx()] =f), “ (a)=p, x(b)=4 
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Example 2.16 


Solution 


2.4.5 


S 
LEO) =f0, x@=p, Fo)=4 

or even such systems as 
LO =f0, xa) =p, Axl) + BE ()=4 


The increased range of possibilities introduced by boundary-value problems almost 
inevitably increases the problems which may arise in their solution. For instance, it may at 
first sight seem that it should also be possible to solve problems with boundary conditions 
consisting of the derivative at both boundaries, such as 


LIX] =f), (a) =p, GX (4) = g 


Things are unfortunately not that simple — as Example 2.16 shows. 


Solve the boundary-value problem 
dx dx dx 
f€*X=4, Sooy=ap, S)= 
ap q Aa as q rea 


Integrating twice easily yields the general solution 
x=20?+At+B 

The boundary conditions then impose 
A=p and 4+A=q 


It is obviously not possible to find a value of A satisfying both these equations unless 
q=p +4. In any event, whether or not p and q satisfy this relation, it is not possible to 
determine the constant B. 


Example 2.16 illustrates the fact that if derivative boundary conditions are to be 
applied, a supplementary compatibility condition is needed. In addition, there may be a 
residual uncertainty in the solution. The complete analysis of what types of boundary 
conditions are allowable for two-point boundary-value problems is beyond the scope of 
this book. Differential equations of orders higher than two increase the range of possi- 
bilities even further and introduce further complexities into the determination of what 
boundary conditions are allowable and valid. 


The method of shooting 


One obvious way of solving two-point boundary-value problems is a form of systematic 
trial and error in which the boundary-value problem is replaced by an initial-value 
problem with initial values given at one of the two boundary points. The initial-value 
problem can be solved by an appropriate numerical technique and the value of whatever 
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Figure 2.22 

The solution of a 
differential equation 
by the method of 
shooting: initial trials. 


xA 
X, () 
qr T4 
pt 
X (6) 
t > 
ie) a b t 


function is involved in the boundary condition at the second boundary point deter- 
mined. The initial values are then adjusted and another initial-value problem solved. 
This process is repeated until a solution is found with the appropriate value at the 
second boundary point. 

As an illustration, we shall consider a second-order boundary-value problem of 
the form 


L[x]=f/(, x(a)=p, x(b)=4 (2.27) 


The related initial-value problem 


LIxJ=/@, x(a) =p, <a) =0 (2.28) 


could be solved as described in Section 2.4.2. Suppose that doing this results in an 
approximate solution of (2.28) denoted by _X,. In the same way, denote the solution of 
the problem 


LIx]=/@, x(a) =p, (a) =i (2.29) 


by_X,. We now have a situation as shown in Figure 2.22. The values of the two solutions 
at the point ¢ = b are X,(b) and X,(b). The original boundary-value problem (2.27) 
requires a value q at b. Since g is roughly three-quarters of the way between X,(b) and 
X,(b), we should intuitively expect that solving the initial-value problem 


L[x] =f), x(a) =p, (a) = 0.75 (2.30) 


will produce a solution with X(6) much closer to g. What we have done, of course, 
is to assume that X(b) varies continuously and roughly in proportion to (dx/dt)(a) 
and then to use linear interpolation to estimate a better value of (dx/df)(a). It is unlikely, 
of course, that X(b) will vary exactly linearly with (dx/d?)(a) so the solution of (2.30), 
call it X;, will be something like that shown in Figure 2.23. The process of linear 
interpolation to estimate a value of (dx/df)(a) and the subsequent solution of the 
resulting initial-value problem can be repeated until a solution is found with a value 
of X(b) as close to g as may be required. This method of solution is known, by an 
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Figure 2.23 


The solution of a 
differential equation 
by the method of 


shooting: first 
refinement. 


2.4.6 


xA 
X, (b) 
ae +4 
7 X; (b) 
pel: 
X, (b) 
| > 
ie) a bh t 


obvious analogy with the bracketing method employed by artillerymen to find their 
targets, as the method of shooting. Shooting is not restricted to solving two-point 
boundary-value problems in which the two boundary values are values of the dependent 
variable. Problems involving boundary values on the derivatives can be solved in an 
analogous manner. 

The solution of a two-point boundary-value problem by the method of shooting 
involves repeatedly solving a similar initial-value problem. It is therefore obvious that 
the amount of computation required to obtain a solution to a two-point boundary- 
value problem by this method is certain to be an order of magnitude or more greater 
than that required to solve an initial-value problem of the same order to the same 
accuracy. The method for finding the solution that satisfies the boundary condition at 
the second boundary point which we have just described used linear interpolation. It is 
possible to reduce the computation required by using more sophisticated interpolation 
methods. For instance, a version of the method of shooting that utilizes Newton—Raphson 
iteration is described in R. D. Milne, Applied Functional Analysis, An Introductory 
Treatment (Pitman, London, 1979). 


Function approximation methods 


The method of shooting is not the only way of solving boundary-value problems numeric- 
ally. Other methods include various finite-difference techniques and a set of methods 
that can be collectively characterized as function approximation methods. In a finite- 
difference method the differential operator of the differential equation is replaced 
by a finite-difference approximation to the operator. This leads to a set of linear 
algebraic equations relating the values of the solution to the differential equation at 
a set of discrete values of the independent variable. Function approximation methods 
include various collocation methods and the finite-element method. In this section we 
shall very briefly outline function approximation methods and give an elementary 
example of the use of a collocation method. It is not appropriate to give an extensive 
treatment of these methods in this book; the reader needing more detail should refer to 
more advanced texts. 

The method of shooting solves a boundary-value problem by starting at one boundary 
and constructing an approximate solution to the problem step by step until the second 
boundary is reached. In contrast with this, function approximation methods find an 
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approximate solution by assuming a particular type or form of function for the solution 
over the whole range of the problem. This function (usually referred to as the trial 
function) is then substituted into the differential equation and its boundary conditions. 
Trial functions always contain some unknown parameters, and, once the function has 
been substituted into the differential equation, some criterion can be used to assign 
values to these initially unknown parameters in such a way as to make the trial function 
as close an approximation as possible to the solution of the boundary-value problem. 

Unless a very fortuitous choice of trial function is made, it is unlikely that it will 
be possible to make the function chosen satisfy the differential equation exactly. If, 
for instance, a trial function depending on some parameters p,, p,, . . . and denoted 
by X(t; P), Po, --- ) is to be used to obtain an approximate solution to the differential 
equation L[x(t)] = 0 then substituting this function into the differential equation results 
in a function 


LIX(G Pi Pa». = 1G Pi, Po» -- +) 


which is called the residual of the equation. Intuitively, it seems likely that making this 
residual as small as possible will result in a good approximation to the solution of the 
equation. But what does making a function as small as possible mean? The most com- 
mon approaches are to make the residual zero at some discrete set of points distributed 
over the range of the independent variable — this gives rise to collocation methods — or 
to minimize, in some way, some measure of the overall size of the residual (for instance, 
the integral of the square of the residual) — this is commonly used in finite-element 
methods. 
Thus, for instance, to solve the boundary-value problem 


LIx)]=0, xaj=q, x(b)=r (2.31) 


we should assume that the trial function X(f), an approximation to x(¢), takes some form 
such as 


X=) p fil) (2.32) 
i=1 
where {p;:i=1,2,...,} is the set of parameters that are to be determined and { f(f) : 
i=1,2,...,n}1is some set of functions of ¢. Substituting the approximation (2.32) into 
the original problem (2.31) gives 


LIS pif) = 10) (2.33a) 
¥ pifila) =4 (2.33b) 
and 

y pi f(b) =r (2.33¢) 


i=1 
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Equations (2.33b, c) express the requirement that the approximation chosen will satisfy 
the boundary conditions of the problem. The function 7(f) in (2.33a) is the residual of 
the problem. Since (2.33b, c) impose two conditions on the choice of the parameters p,, 


Po, -+-5 DP, we need another n — 2 conditions to determine all the p,. For a collocation 
solution this is done by choosing n — 2 values of tf such thata <t,<<t,<<...<t,,<b 
and making 77(t,) = 0 fork =1, 2,...,n—2. Thus we have the n equations 
LIS pifi(t)|=0 (kK=1,2,...,n-2) (2.34a) 
r= 
> Pi fla) =4 (2.34b) 
>) Pi f(b) =r (2.34) 


for the n unknown parameters p,, p>, ...,/,- In general, these equations will be nonlinear 
in the p,, but if the operator L is a linear operator then they may be rewritten as 


¥ pill fits)] = 0 WS IZ ocep = 2) (2.35a) 
Sy Di fla) =4 (2.35b) 
Seas (2.35¢c) 
and are linear in the p;. They therefore constitute a matrix equation for the p;: 
LIK) = LAGI Lima] |f | fol 
LIA(t)] LIfA(t)] Lifi(t)] || pe 0 
LIA()] = LLAG)I LIf(ts)] || Ps 0 
pe) (2.36) 
LI A(t-2)] LI A(-2)] LES Ctn-2)] Pn-2 
fi(a) f(a) Sila) Pr 
fi(d) f(b) Sil b) Pull es 


This matrix equation can, of course, be solved by any of the standard methods of linear 
algebra. If the operator L is nonlinear then (2.34) cannot be expressed in the form 
(2.35). The equations (2.34) may still be solved for the coefficients p,, but the solution 
of nonlinear equations is, in general, a much more difficult task than the solution of 
linear ones. 

The choice of the functions f;(4) and the collocation points ¢, greatly affect the accur- 
acy and speed of convergence of the solution. (The speed of convergence in this context 
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Example 2.17 


Solution 


is usually measured by the number of terms it is necessary to take in the approxima- 
tion (2.32) in order to achieve a solution with a specified accuracy.) Example 2.17 shows 
a simple application of collocation methods to the solution of a second-order boundary- 
value problem. 


Solve the boundary-value problem 


d’x dx _ _ _ 
Sore te 0, x(0)=0, x2)=1 (2.37) 
dt dt 


using a collocation method with 


X= Spt 
i=l 


The differential operator in this case is linear, so we may construct the matrix equation 
equivalent to (2.36). With the given approximation, we have 
[((i-1)G-2+te')+P]t° (i>3) 
LIAM =Lit') = 4 (e+) (i=2) 
1 G=1) 
We shall choose the collocation points to be equally spaced over the interior of the 


interval [0, 2]. Thus, for 7 = 5 say, we need three collocation points, which would be 
0.5, 1.0 and 1.5. We should therefore obtain the matrix equation 


LLfi(0.5)] LLA(0.5)] LLA(0-5)] LLA(0.5)] LLA(0-5)] |} pr 0 
L{fC.0)] LLAG.0)] LIAC.0)] LIfZAG.0)] LLAC.0)] |} pe 0 
LIAG-5S)] LIAG-5S)] LIAC-5S)] LIf0-5)] LLfC.5)]]] ps | ~ | 0 
1 0 0 0 0 Ps 0 
1 2 4 8 16 Ds 1 


Computing the numerical values of the matrix elements yields the matrix equation 


1.000 2.149 3.899 4362 3.887 || p, 0.000 | 
1.000 3.718 8.437 15.155 23.873 || p,| | 0.000 
1.000 5.982 17.695 42.626 92.565 || p; | =| 0.000 
1.000 0.000 0.000 0.000 0.000 || p,| | 0.000 
1.000 2.000 4.000 8.000 16.000|| p;| | 1.000 | 


whose solution is 
p=[0.000 2.636 -1.912 0.402 0.010]' 


Figure 2.24 shows the solutions X,, X;, X, and X;,. As we should intuitively expect, 
taking more terms in the approximation for x(t) causes the successive approximations 
to converge. In Figure 2.25 the approximations X, and X, are compared with a solution 
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Method 
of 
0.5 5 shooting 
> -—> 
O 0.5 1.0 1.5 2.0 t O 0.5 1.0 1.5 2.0 ¢ 


Figure 2.24 A collocation solution of (2.37). 


Example 2.18 


Figure 2.25 Comparison of the collocation solutions 
with the solution by the method of shooting. 


to the problem (2.37) obtained by the method of shooting using a second-order Runge— 
Kutta integration method. The step size used for the method-of-shooting solution was 
estimated, using the technique introduced in Section 2.3.6, to yield a solution accurate 
to better than 3.5 x 10°. On this graph the solution Y, was indistinguishable from the 
method-of-shooting solution. 


Although Example 2.17 gave reasonably good accuracy from a relatively small 
number of terms in the function X,,(A), difficulties do arise with collocation methods when 
straightforward power-series approximations like this are used. It is more normal to use 
some form of orthogonal polynomials, such as Tchebyshev or Legendre polynomials, 
for the (4). In appropriate cases f;(t) = sin it and cos it are also used. The reader is 
referred to more advanced texts for details of these functions and their use in collocation 
methods. 

Although they are rather more commonly used for problems involving partial dif- 
ferential equations, finite-element methods may also be used for ordinary differential 
equation boundary-value problems. The essential difference between finite-element 
methods and collocation methods of the type described in Example 2.17 lies in the type 
of functions used to approximate the dependent variable. Finite-element methods use 
functions with localized support. By this, we mean functions that are zero over large 
parts of the range of the independent variable and only have a non-zero value for some 
restricted part of the range. A complete approximation to the dependent variable may 
be constructed from a linear sum of such functions, the coefficients in the linear sum 
providing the parameters of the function approximation. 


A typical simple set of functions with localized support that are often used in the finite- 
element method are the ‘witch’s hat’ functions. For a one-dimensional boundary-value 
problem, such as (2.31), the range [a, b] of the independent variable is divided into a 
number of subranges [f, ¢,], [4, 61, ..-» [4,1, ¢,] with t) = a and t, = b. We then define 
functions 
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Figure 2.26 
The ‘witch’s hat’ 
functions. 


Figure 2.27 

The construction 
of a continuous 
piecewise-linear 
approximation 
function from 
‘witch’s hat’ 
functions. 


xA 

a /- 7 SiO 

| | > 

O aot th an aes tb t 
xA 

(4, Pi) 

(to, P2) 
> 

@) 


t—t) 
te [tot 
FS (re [ntl 
(t) =$ t.,-t 
i) Ht (té€ [ti tisi]) 
lai — 4; 
0 (¢@ [t-1 tid 


The function f(4) has support (that is, its value is non-zero) only on the interval [f,,, ¢,,,]. 
Figure 2.26 shows the form of the functions /(¢). An approximation to the solution of a 
boundary-value problem can be formed as 


X(N) =P Pi f(D) (2.38) 


k=0 


This equation defines a function that is piecewise-linear and continuous on the range 
[a, b] as illustrated in Figure 2.27. 


The finite-element method provides a general framework for using functions with 
localized support to construct an approximation to the whole solution. One advant- 
age of using such functions is that the user can, to a considerable extent, tailor the 
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approximation used to the properties of the physical problem. If the problem is expected 
to give rise to very rapid changes in some region then more functions with local support 
in that area can be used. In regions where the solution is expected to change relatively 
slowly fewer functions may be used. In Figure 2.27, for instance, the division of the 
interval [a, b] into subregions is shown as being finer near ¢, and coarser near ¢,,. This 
property of functions with local support gives the finite-element method considerable 
advantages over collocation methods (which use functions defined over the whole 
range of the problem) and over finite-difference methods. 

Just as for the function approximation method illustrated in Example 2.17, the finite- 
element method requires that some criterion be chosen for determining the values of 
the unknown parameters in the approximation (2.38). A variety of criteria are commonly 
used, but we shall not describe these in detail in this section. The use of the finite-element 
method for obtaining numerical solutions of partial differential equations is described 
in Section 9.6. 


2.5 Engineering application: SwE eee mele 


The simple pendulum has been used for hundreds of years as a timing device. A 
pendulum clock, using either a falling weight or a clockwork spring device to provide 
motive power, relies on the natural periodic oscillations of a pendulum to ensure good 
timekeeping. Generally we assume that the period of a pendulum is constant regardless 
of its amplitude. But this is only true for infinitesimally small amplitude oscillations. In 
reality the period of a pendulum’s oscillations depends on its amplitude. In this section 
we will use our knowledge of numerical analysis to assist in an investigation of this 
relationship. 

Figure 2.28 shows a simple rigid pendulum mounted on a frictionless pivot swinging 
in a single plane. By resolving forces in the tangential direction we have, following the 
classical analysis of such pendulums, 


ma = -mg sin@ 
t 


Figure 2.28 A simple 
pendulum. 


that is, 


= + £ sing=0 (2.39) 
a 


For small oscillations of the pendulum we can use the approximation sin 0 ~ @ so the 
equation becomes 


2 
eee 0 (2.40) 
dt a 


which is, of course, the simple harmonic motion equation with solutions 
6=A cos (22) +B sin( /£r) 
a a 
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Hence the period of the oscillations is 27,/(a/g) and is independent of the amplitude of 
the oscillations. 

In reality, of course, the amplitude of the oscillations may not be small enough for 
the linear approximation sin 6 = @ to be valid, so it would be useful to be able to solve 
(2.39). Equation (2.39) is nonlinear so its solution is rather more problematical than 
(2.40). We will solve the equation numerically. In order to make the solution a little 
more transparent we will scale it so that the period of the oscillations of the linear 
approximation (2.40) is unity. This is achieved by setting t = 27(a/g) t. Equation (2.39) 
then becomes 

2 
_ + 4n’ sin @= 0 (2.41) 
dt 


For an initial amplitude of 30°, the pseudocode algorithm shown in Figure 2.29, which 
implements the fourth-order Runge—Kutta method described in Section 2.3.8, produces 
the results ©(6.0) = 23.965 834 using a time step of 0.05 and ©(6.0) = 24.018 659 with 
a step of 0.025. Using Richardson extrapolation (see Section 2.3.6) we can predict that 
the time step needed to achieve 5 dp of accuracy (i.e. an error less than 5 x 107°) with 
this fourth-order method is 


1/4 


0.000 005 x (2° = 1) x.0.025 = 0.0049 
| 23.965 834 - 24.018 659| 


repeating the calculation with time steps 0.01 and 0.005 gives ©(6.0) = 24.021 872 7 and 
(6.0) = 24.021 948 1 for which Richardson extrapolation implies an error of 5 x 10° 
as predicted. 


These results could also have been obtained using MAPLE as shown by the follow- 
ing worksheet: 

SS Cleesis eaClalibie (Ox (1c) pie SA) se aL Assim (sx (ie) ) = 

S dats s=(0)=60/1S80" bi , Ds) (0) =O 

> sol:=dsolve({deqsys, inits}, numeric,method=classical 


[rk4] , output=listprocedure, stepsize=0.005); 
= socg=Co (2, SOL 2])) esec(G) genvalit (Sec (6) -1LSO/ 2x) 5 


As a check we can draw the graph of |© 0,(7) — ©o o9s(T)|/15, shown in Figure 2.30. 
This confirms that the error grows as the solution advances and that the maximum error 
is around 7.5 x 10°. 

What we actually wanted is an estimate of the period of the oscillations. The most 
satisfactory way to determine this is to find the interval between the times of successive 
zero crossings. The time of a zero crossing can be estimated by linear interpolation between 
the data points produced in numerical solution of the differential equation. At a zero 
crossing the successive values of © have the opposite sign. Figure 2.31 shows a modified 
version of the main part of the algorithm of Figure 2.29. This version determines the times 
of successive positive to negative zero crossings and the differences between them. 

Figure 2.32 shows some results from a program based on the algorithm of Figure 2.31; 
it is evident that the period has been determined to 6 sf accuracy. Figure 2.33 has been 
compiled from similar results for other amplitudes of oscillation. 
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Figure 2.29 

A pseudocode 
algorithm for solving 
the nonlinear pendulum 
equation (2.41). 


tol — 0.00001 
t_start — 0 
t_end < 6 
write(vdu, ‘Enter amplitude => ’) 
read(keyb, x0) 
x_start — pi*x0/180 
v_start + 0 
write(vdu, ‘Enter stepsize => ’) 
read(keyb,h) 
write(vdu,t_start,“ ”,deg(x_start) ) 
t < t_start 
ac exees tanh 
v < v_Start 
repeat 
rk4(x,v,h — xn,vn) 
x<xn 
vevn 
te tth 
until abs(t — t_end) < tol 
write(vdu,t, ‘ ”,deg(x)) 


procedure rk4(x,v,h > xn,vn) 

cll < hefl(x,v) 

c21 < h*f2(x,v) 

cl2 < hefl(x +.c11/2,v + c21/2) 

c22 <— h*f2(x +.c11/2,v + c21/2) 

cl3 — h*fl(x + c12/2,v + c22/2) 

c23 <— h*f2(x + c12/2,v + c22/2) 

cl4 < hefl(x + c13,v + c23) 

024 < h*f2(x + c13,v + c23) 

xn — x + (cll + 2*(c1l2 +13) +. c14)/6 

vn — v+(c21 + 2*(c22 + c23) + c24)/6 
endprocedure 


procedure fl(x,v — fl) 
flev 
endprocedure 


procedure f2(x,v —> f2) 
f2 — —4*pi*pi*sin (x) 
endprocedure 


procedure deg(x — deg) 
deg — 180+x/pi 
endprocedure 


Some spring-powered pendulum clocks are observed to behave in a counter-intuitive 
way — as the spring winds down the clock gains time where most people intuitively 
expect it to run more slowly and hence lose time. Figure 2.33 explains this phenom- 
enon. The reason is that, in a spring-powered clock, the spring, acting through the 
escapement mechanism, exerts forces on the pendulum which, over each cycle of oscil- 
lation of the pendulum, result in the application of a tiny net impulse. The result is that 
just sufficient work is done on the pendulum to overcome the effects of bearing friction, 
air resistance and any other dissipative effects, and to keep the pendulum swinging with 
constant amplitude. But, as the spring unwinds the force available is reduced and the 
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Figure 2.30 8.0E-6 
Error in solution 
of equation (2.41) 6.0E-6 L\ 
using algorithm 
(2.30) with 
h = 0.005. eS | | 
2.0E-6 L\ 
3 
E  0.0E+0 
2 0.00 0 2,00 3.00 4.00 5 6.00 
2.0E-6 
—4.0E-6 y 
—6.0E-6 Ti 
—8.0E-6 
Time 
Figure 2.31 
Modification of ae ue . 
pseudocode algorithm af 7 se 
to find the period ee : ‘ : 
Af eacillahonsot write(vdu, ‘Enter amplitude => ’) 
equation (2.41). Readlkey05)) 
x_start — pi*x0/180 
v_start — 0 
write(vdu, ‘Enter stepsize => °) 
read(keyb,h) 
write(vdu,t_start,“ ” ,deg(x_start) ) 
t < t_start 
xe xen taht 
v < v_start 
t_previous_cross < t_start 
repeat 


tk4(x,v,h — xn,vn) 

if(xn*x < 0) and (x > 0) then 
t_cross < (t*xn — (t + h)*x)/(xn-x) 
write(vdu,t_cross,‘ ”,t_cross — t_previous_cross) 
t_previous_cross < t_cross 

endif 

x xn 

veyn 

t<tth 

until abs(t — t_end) < tol 


impulse gets smaller. The result is that, as the clock winds down, the amplitude of 
oscillation of the pendulum decreases slightly. Figure 2.33 shows that as the amplitude 
decreases the period also decreases. Since the period of the pendulum controls the 
speed of the clock, the clock runs faster as the period decreases! Of course, as the clock 
winds down even further, the spring reaches a point where it is no longer capable of 
applying a sufficient impulse to overcome the dissipative forces, the pendulum ceases 
swinging and the clock finally stops. 
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Figure 2.32 Periods of 


successive oscillations Time of crossing Period of last cycle 
of equation (2.41), 
©, = 30°, h = 0.005. 0.254 352 13 
1.271 761 06 1.017 408 93 
2.289 169 73 1.017 408 67 
3.306 578 68 1.017 408 95 
4.323 987 34 1.017 408 66 
5.341 396 30 1.017 408 96 
Figure 2.33 1.2 
Variation of period of 
oscillations of equation 
(2.41) with amplitude. 
1.15 
=I 
g 
Re 
3B 
eu 
° 
uo] 
g 
B) 
[aw 
1.05 
1 Presi 


0 10 20 30 40 50 60 70 80 90 
Amplitude of oscillation 


(2) The periods of the oscillations can also be measured using MAPLE. The procedure 
fsolve finds numerically the roots of a function. The output of the procedure 
dsolve is a function so we can use fsolve to find the zeros of that function, as in 
the following MAPLE worksheet. Note that the period of successive cycles is found 
more accurately and consistently using MAPLE. This is because the procedure 
fsolve uses a higher-order method to locate the zeros of the function rather than 
the linear interpolation method outlined in the algorithm in Figure 2.31. 


> print levels): 
= for a tmenm 1 te 6 choy 
els Ssesiollye(Sex (eo) SO, 18, (=il)) o 5 (140.99) 2 
tA gairsol wa (see (S01, bas (0.99) )) 2 
joneaumieit (S52), Vie, ee oo Pie, el Te, Vin” tee, etc 1.) 9 
end do; 


yee Aaedlntlalarcar)) elie heating of an electrical fuse 


The electrical fuse is a simple device for protecting an electrical apparatus or circuit 
from overload and possible damage after the failure of one or more components in the 
apparatus. A fuse is usually a short length of thin wire through which the electrical current 
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powering the apparatus flows. If the apparatus fails in such a way as to draw a dangerously 
increased current, the fuse wire heats up and eventually melts thus disconnecting the 
apparatus from the power source. In order to design fuses which will not fail during 
normal use but which will operate reliably and rapidly in abnormal circumstances we 
must understand the heating of a thin wire carrying an electrical current. 

The equation governing the heat generation and dissipation in a wire carrying an 
electrical current can be formulated as 

2 
—knr L424 onrn(r-7,)* = PL, (2.42) 
dx Tr 

where 7 is the temperature of the fuse wire, x is the distance along the wire, k is the 
thermal conductivity of the material of which the wire is composed, r is the radius of 
the wire, h is the convective heat transfer coefficient from the surface of the wire, 7, is 
the ambient temperature of the fuse’s surroundings, @ is an empirical constant with a 
value around 1.25, / is the current in the wire and p is the resistivity of the wire. Equa- 
tion (2.42) expresses the balance, in the steady state, between heat generation and heat 
loss. The first term of the equation represents the transfer of heat along the wire by 
conduction, the second term is the loss of heat from the surface of the wire by convec- 
tion and the third term is the generation of heat in the wire by the electrical current. 

Taking 6 =(T— T.) and dividing by knr’, (2.42) can be expressed as 


LT ae (2.43) 


Letting the length of the fuse be 2a and scaling the space variable, x, by setting x = 2aX, 
(2.43) becomes 
2 


dg’ _ 8a°h yg __4a°pl’ 

dX? = kr knr’ 

The boundary conditions are that the two ends of the wire, which are in contact with the 
electrical terminals in the fuse unit, are kept at some fixed temperature (we will assume 
that this temperature is the same as T.). In addition, the fuse has symmetry about its 
midpoint x = a. Hence we may express the complete differential equation problem as 


2 2 2 2 
GO _ Sah ge __Sapl 0(0) = 0, Co iie0 (2.44) 
dx kr kur dx 


Equation (2.44) is a nonlinear second-order ordinary differential equation. There is 
no straightforward analytical technique for tackling it so we must use numerical means. 
The problem is a boundary-value problem so we must use either the method of shooting 
or some function approximation method. Figure 2.34 shows a pseudocode algorithm for 
this problem and Figure 2.35 gives the supporting procedures. The procedure ‘desolve’ 
assumes initial conditions of the form 6(0) = 0, d@/d.X(0) = 64 and solves the differential 
equation using the third-order predictor—corrector method (with a single fourth-order 
Runge-Kutta step to start the multistep process). The main program uses the method of 
regula falsa to iterate from two starting values of 0) which bracket that value of 0) 
corresponding to d@/dX(1) = 0 which we seek. 

Figure 2.36 shows the result of computations using a program based on the algorithm 
in Figure 2.34. Taking the values of the physical constants as h = 100 Wm*K"', a=0.01 m, 
k=63Wm'K"™, p= 16x 10%Q mandr=5 x 10‘ m, and taking / as 20 amps and 
40 amps, gives the lower and upper curves in Figure 2.36 respectively. 
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Figure 2.34 
Pseudocode algorithm Ora ones 
: : kappa <— 63 
for solving equation 
(2.44) r<5e-4 
- a< le-2 
hh < le2 
ic 20 


pconst — 8*hh*a*a/(kappar) 
qconst <— 4«a*a*rho+i+i/(kappa*pi*pi*r+r#r+r) 
tol < le-5 
x_start — 0.0 
x_end < 1.0 
theta_start — 0.0 
write(vdu, ‘Enter stepsize -->’) 
read(keyb,h) 
write(vdu, ‘Enter lower limit -->’) 
read(keyb,theta_dash_low) 
write(vdu, ‘Enter upper limit -->’) 
read(keyb,theta_dash_high) 
desolve(x_start,x_end,h,theta_start,theta_dash_low — th,ql) 
desolve(x_start,x_end,h,theta_start,theta_dash_high — th,qh) 
repeat 
theta_dash_new < (qh*theta_dash_low — ql*theta_dash_high)/(qh — ql) 
desolve (x_start,x_end,h,theta_start,theta_dash_new — th,qn) 
if ql*qn>0 then 
ql < qn 
theta_dash_low < theta_dash_new 
else 
qh < qn 
theta_dash_high < theta_dash_new 
endif 
until abs(qn) < tol 
write(vdu,th,qn) 


procedure desolve(x_0,x_end,h,v1_0,v2_0 — v1_f,v2_f) 
xe x0 
vl_o¢e vl_0 
v2_0¢ v2_0 
tk4(x,vl_o0,v2_0,h > v1,v2) 
x <—xth 
repeat 
pc3(x,vl_o,v2_0,v1l,v2,h, > vl_n,v2_n) 
vl_o¢vl 
v2_0< v2 
vl<vl_n 
v2<¢ v2_n 
x xth 
until abs(x — x_end) < tol 
vl_f<evl 
v2_f< v2 
endprocedure 


Evidently at 20 amps the operating temperature of the middle part of the wire is 
about 77° above the ambient temperature. If the current increases to 40 amps the 
temperature increases to about 245° above ambient — just above the melting point of tin! 
The procedure could obviously be used to design and validate appropriate dimensions 
(length and diameter) for fuses made from a variety of metals for a variety of applica- 
tions and rated currents. 
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Figure 2.35 
Subsidiary procedures 
for pseudocode 
algorithm for solving 
equation (2.44). 


Figure 2.36 
Comparison of 
temperatures in a fuse 
wire carrying 20 amps 
and 40 amps. 
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procedure rk4 (x,vl,v2,h > vin,v2n) 
cll < hefl(x,vl,v2) 
c21 < h*f2(x,vl,v2) 
cl2 < hefl(x + h/2,vl + .¢11/2,v2 + c21/2) 
c22 — hef2(x + h/2,vl + .c11/2,v2 + c21/2) 
c13 — hefl(x + h/2,vl + c12/2,v2 + c22/2) 
c23 + h*f2(x + h/2,vl + ¢12/2,v2 + c22/2) 
cl4 < hefl(x +h,vl + c13,v2 + c23) 
c24 — h*f2(x +h,vl + c13,v2 + c23) 
vin vl + (cll + 2%(c12 +.c13) + c14)/6 
v2n < v2 + (c21 + 2%(c22 + c23) + c24)/6 
endprocedure 


procedure pc3(x, vl_o,v2_0,vl,v2,h > v1_n,v2_n) 
vl_p < vl + h*(3*fl(x,v1,v2) — f1(x — h,v1_0,v2_0))/2 
v2_p © v2 + h+(3*f2(x,vl,v2) — f2(x — h,vl_o,v2_0))/2 
vil_n¢ vl + h*(5*fl(x + h,vl_p,v2_p) 
+ 8*f1(x,vl,v2) — f1(x — h,vl_o,v2_0))/12 
v2_n © v2 + h#(5*f2(x + h,vl_p, v2_p) 
+ 8*f2(x,v1,v2) — f2(x — h,vl_o,v2_0))/12 
endprocedure 


procedure f1(x, theta,theta_dash — fl) 
fl < theta_dash; 
endprocedure 


procedure f2(x,theta,theta_dash — f2) 
if theta < tol then 
f2 < —qconst 
else 
f2 < pconst*exp(In (theta)*1.25) — qconst 
endif 
endprocedure 


250 


200 


—_ 
un 
Co 


— 
So 
o 


Temperature 


50 


0 0.1 0.2 0.3 0.4 0.5 


0.6 0.7 0.8 0.9 1 


Distance along fuse 


www.2Ofile.org 


1783 NUMERICAL SOLUTION OF ORDINARY DIFFERENTIAL EQUATIONS 


(2) The differential equation problem to be solved in this application is a boundary- 
value problem rather than an initial-value problem. MAPLE’s dsolve procedure 
can readily handle this type of problem. The following MAPLE worksheet repro- 
duces the temperature profiles shown in Figure 2.36. 

S Gleciswysis elie ic (elieice. (se) 24,28) el Ala / 
(ets) theta (s)“allpna=—4 a" 24*rorn 2)/ (k* Pro Are A i 
S iMaliegi A Stelasice (0) 0), i(iclasivel) (1) 05 
S aljomegail  A5pls=l0Osas=0, 01 pikeeaGs pices = Ge) pies aul s 
1gS20)¢ 


> soll:=dsolve({degqsys, inits}, 
numeric, output=listprocedure,maxmesh=512) ; 


S esd 
> sol2:=dsolve({deqsys, inits}, 
numeric, output=listprocedure,maxmesh=512) ; 
S Oo(2, S012] ) (1) eea(2,sol2 [2]1)) (ye 
= plot ([oo(2, soli [2 ])) ,co(2,sol2 21) ],0.si)e 


To find a numerical solution of a second-order differential equation using 
MATLAB, the user must first carry out the transformation to a set of two first-order 
equations; MATLAB, unlike MAPLE, cannot complete this stage internally. Then 
the following MATLAB M-file solves the differential equation and reproduce the 
temperature profiles shown in Figure 2.36. 


function engineering_app2 

a0, Vi gi=100pik=63 pw=S5e-4 palhoiesil . 25 ¢roOSloe-sp1=20p 
Ssolimiic = loywiolimailc (limspace(0, 1,10), [40 O.51))) 5 
soll = bvp4c(@odefun, @bcfun,solinit) ; 

LeAds 

SOl2 = lowiole ((Cocleirum, Clacituin, So@lliianic)) p 

x = llingioace (0, iL) ¢ 

vil = Clerval (iSs@ll il ,s2)) ¢ 

V2 = Cevell (Ss@l2 ,s2) 7 

DOloe (2,71 (1, 2) pe ve(,s))e 

gril (aL, Ov!) 

We CL, LOO) 


EUNGeELon GCydx = oderun (x,y) 
Ghycbe = || wi) 
Shey 2lay? (hese) 7 (AL) “eulljolna aber 2 ieee al 2) / (sia 2 te 4) || 
end 
function res = bcfun(ya,yb) 
resi = | we (i) 
Viol) Is 
end 
end 
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2.7 Review exercises (1-12) 


il 


2 


3 


4 


5 


Find the value of X(0.5) for the initial-value 
problem 


dx 
= ne8, ONS I 
ay vx, x(0) 
using Euler’s method with step size h = 0.1. 


Find the value of X(1.2) for the initial-value 
problem 


a = —e", x1)=1 


using Euler’s method with step size h = 0.05. 


Solve the differential equation 
or 1, x)=! 
dt ie aR L 


to find the value of X(0.4) using the Euler method 
with steps of size 0.1 and 0.05. By comparing the 


two estimates of x(0.4) estimate the accuracy of the 


better of the two values which you have obtained 


and also the step size you would need to use in order 


to calculate an estimate of x(0.4) accurate to two 
decimal places. 


Solve the differential equation 


=sin(t?), x(0)=2 


to find the value of X(0.25) using the Euler method 
with steps of size 0.05 and 0.025. By comparing 


the two estimates of x(0.25) estimate the accuracy 
of the better of the two values which you have 
obtained and also the step size you would need to 
use in order to calculate an estimate of x(0.25) 
accurate to three decimal places. 


Let X,, X, and X; denote the estimates of the 


e) function x(f) satisfying the differential equation 


= = ot+), x()=2 


which are calculated using the second-order 
predictor—corrector method with steps of 0.1, 0.05 


and 0.025 respectively. Compute X,(1.2), X,(1.2) 
and X;(1.2). Show that the ratio of |X, — X,| and 


www.2Ofile.org 


|X, — X,| should tend to 4: 1 as the step size 
tends to zero. Do your computations bear out 
this expectation? 


Compute the solution of the differential equation 
Hae“, x(0)=5 


for x = 0 to 2 using the fourth-order Runge-Kutta 
method with step sizes of 0.2, 0.1 and 0.05. 
Estimate the accuracy of the most accurate of 
your three solutions. 


In a thick cylinder subjected to internal pressure 
the radial pressure p(r) at distance r from the axis 
of the cylinder is given by 


perce =2a =p 
dr 


where a is a constant (which depends on the 
geometry of the cylinder). 

If the stress has magnitude py at the inner wall, 
r=Tfo, and may be neglected at the outer wall, 
r=r,, show that 


2 2 
i ie 
por= out (5-1) 


Fi ako 


If 7) = 1, r, =2 and py = 1, compare the value 
of p(1.5) obtained from this analytic solution 
with the numerical value obtained using the 
fourth-order Runge—Kutta method with step size 
h=0.5. (Note: with these values of ro, 7, and Pp, 
a=-1/3). 


Find the values of X(t) for t up to 2 where X(t) 
is the solution of the differential equation 
problem 


3 ae 
+ (3) +4(Se)'- ix =sint, 
dt dr dt 


2 
x1) =02, %ay=1, 44a) =0 
dt dt 


using the Euler method with steps of 0.025. 
Repeat the computation with a step size of 
0.0125. Hence estimate the accuracy of 

the value of X(2) given by your solution. 
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Find the solution of the differential equation problem Investigate the properties of the Van der 
5 Pol oscillator. In particular show that the 
dx a (x7 = 1% + 40x =0, oscillator shows limit cycle behaviour (that 
‘ dt is, the oscillations tend to a form which is 


independent of the initial conditions and depends 
only on the parameter j). Determine the 
dependence of the limit cycle period on ju. 


= dx (9) — 
x(0) = 0.02, (0) =0 


using the second-order predictor—corrector 
method. Hence find an estimate of the value 12 


; (Extended, open-ended problem.) The equation 
of x(4) accurate to four decimal places. 


=] of simple harmonic motion 


Find the solution of the differential equation problem & 
Laie 
. ee = +4x=0 
2 3 = 
Coe oe +4(&) -ix=sint, & 
dr |dt dt 


is generally used to model the undamped 
oscillations of a mass supported on the end of 

a linear spring (that is, a spring whose tension is 
strictly proportional to its extension). Most real 
springs are actually nonlinear because as their 
extension or compression increases their 
stiffness changes. This can be modelled by 

the equation 


2 
x)=-1, ®ay=1, ay=2 
dt dt 


using the fourth-order Runge-Kutta method. 
Hence find an estimate of the value of x(2.5) 
accurate to four decimal places. 


(Extended, open-ended problem.) The second- 


order, nonlinear, ordinary differential equation dx 
F — + 4n°(1 + Bx’)x = 0 
dt 
ee ye 0 
dt dt 


For a ‘hard’ spring stiffness increases with 


governs the oscillations of the Van der Pol displacement (8 > 0) and a soft spring’s stiffness 
oscillator. By scaling the time variable the decreases (8 < 0). Investigate the oscillations 
equation can be reduced to of a mass supported by a hard or soft spring. In 
particular determine the connection between 
dx + U(x’ - 1 +(2n)’x =0 the frequency of the oscillations and their 
: dt amplitude. 


www.2Ofile.org 


\ Bea Vector Calculus 


Chapter 3 Contents 


3.1 Introduction 182 
3.2 Derivatives of a scalar point function 199) 
3.3 Derivatives of a vector point function 203 
3.4 Topics in integration 214 
3.5 Engineering application: streamlines in fluid dynamics 248 
3.6 Engineering application: heat transfer 250) 
3.7 Review exercises (1-21) 254 


www.2Ofile.org 


182 VECTOR CALCULUS 


Figure 3.1 
Elementary 
vector algebra. 


op 


Introduction 


In many applications we use functions of the space variable r = xi + yj + zk as models 
for quantities that vary from point to point in three-dimensional space. There are two types 
of such functions. There are scalar point functions, which model scalar quantities like 
the temperature at a point in a body, and vector point functions, which model vector 
quantities like the velocity of the flow at a point in a liquid. We can express this more 
formally in the following way. For each scalar point function fwe have a rule, u = /(r), 
which assigns to each point with coordinate r in the domain of the function a unique 
real number w. For vector point functions the rule v = F(r) assigns to each r a unique vector 
v in the range of the function. Vector calculus was designed to measure the variation of 
such functions with respect to the space variable r. That development made use of the ideas 
about vectors (components, addition, subtraction, scalar and vector products) described 
in Chapter 4 of Modern Engineering Mathematics and summarized here in Figure 3.1. 


b —b 
b 
i a+b a—b a 
addition subtraction 


components 


b 
6 
a 


a+b =|a\|b| cos 0 
scalar product 


c=axb 
lel = |a||b| sin 8 
vector product 


In component form if a = (a, a>, a,) and b = (b,, b,, b;) then 
atb=(a,tb,, a, + b,, a; + bs) 


a:b =(a,b,+ ab, + a;b,;)=b-a 


i j ik 
axb=ja,; a, a,)=—-bxa 
b, by bs 


= (a,b, — bya,, b,a, — a,b, ab, — b,az) 
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Figure 3.2 
Level surfaces 


3.1.1 


of f(r) = 2, 2, -l)-r 


=2x+2y—z. 


The recent development of computer packages for the modelling of engineering 
problems involving vector quantities has relieved designers of much tedious analysis 
and computation. To be able to use those packages effectively, however, designers need 
a good understanding of the mathematical tools they bring to their tasks. It is on that 
basic understanding that this chapter focuses. 


Basic concepts 


re 


cs 
oa 


We can picture a scalar point function f(r) by means of its level surfaces f(r) = constant. 
For example, the level surfaces of f(r) = 2x + 2y — z are planes parallel to the plane 
z= 2x + 2y, as shown in Figure 3.2. On the level surface the function value does not 
change, so the rate of change of the function will be zero along any line drawn on the 
level surface. An alternative name for a scalar point function is scalar field. This is in 
contrast to the vector point function (or vector field). We picture a vector field by its 
field (or flow) lines. A field line is a curve in space represented by the position vector 
r(t) such that at each point of the curve its tangent is parallel to the vector field. Thus 
the field lines of F(r) are given by the differential equation 

dr _ F _ 
—=Fi(r), where r(t)) =r 

dt 
and ry is the point on the line corresponding to ¢ = f). This vector equation represents 
the three simultaneous ordinary differential equations 


& = Pe,» 


2 O(x, ys Z), 


e = R(x, y, 2) 
where F = (P, Q, R). 
Modern computer algebra packages make it easier to draw both the level surfaces of 
scalar functions and the field lines of vector functions, but to underline the basic ideas 
we shall consider two simple examples. 
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Example 3.1 — Sketch 
(a) the level surfaces of the scalar point function f(r) = ze™; 


(b) _ the field lines of the vector point function F(r) = (—y, x, 1). 


Solution (a) Consider the level surface given by f(r) = c, where c is a number. Then 
ze’ =c and soz=ce”. For c, x and y all positive we can easily sketch part of 
the surface as shown in Figure 3.3(a), from which we can deduce the appearance 
of the whole family of level surfaces. 


Figure 3.3 (a) Level 
surfaces of f(r) =ze~; 
(b) field lines of 


F(r)=(~, x, 1). 


(b) For the function F(r) = (—y, x, 1) the field lines are given by 
# = (y,%,1) 


that is, by the simultaneous differential equations 


dx dy_,  dz_ 
dt 


a dé 
The general solution of these simultaneous equations is 
x(t)=Acost+Bsint, y(t)=—-Bcost+Asint, z(t)=t+C 


where A, B and C are arbitrary constants. Considering, in particular, the field line 
that passes through (1, 0, 0), we determine the parametric equation 


(x(t), W(t), 2(t)) = (cost, sin t, ft) 


This represents a circular helix as shown in Figure 3.3(b), from which we can 
deduce the appearance of the whole family of flow lines. 
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In MATLAB a level surface may be drawn using the ezsurf function. Using the 
Symbolic Math Toolbox the commands: 

vm << Ww 2 © 

tore € = [il 2 3] 


BZ = @vengo (sey) e 

ezusuicr (za, [0, 2, O, 21) 
hold on 

end 


will produce three of the level surfaces of z = e~*” on the same set of axes. The 
surfaces may also be produced in MAPLE using the ezsurf function. The field 
lines may be plotted in MATLAB using the streamline function. 


To investigate the properties of scalar and vector fields further we need to use the 
calculus of several variables. Here we shall describe the basic ideas and definitions 
needed for vector calculus. A fuller treatment is given in Chapter 9 of Modern Engineer- 
ing Mathematics. 

Given a function f(x) of a single variable x, we measure its rate of change (or 
gradient) by its derivative with respect to x. This is 


df — £t) = jim L4 +49 —f) 
dx Ax0 Ax 


However, a function f(x, y, z) of three independent variables x, y and z does not have a 
unique rate of change. The value of the latter depends on the direction in which it is 
measured. The rate of change of the function (x, y, z) in the x direction is given by its 
partial derivative with respect to x, namely 


One lim LOX + A™ Y, 2) — (Oe y, 2) 
Ax 


X Ax>0 


This measures the rate of change of f(x, y, z) with respect to x when y and z are held 
constant. We can calculate such partial derivatives by differentiating f(x, y, z) with 
respect to x, treating y and z as constants. Similarly, 


Of _ lim £22 y+Ay, z)-f(, y; Zz) 


OV ayo Ay 


and 


Of _ lim 4% y, z+ Az) - fix, y, 2) 
Az 


'Z  Az—>0 


define the partial derivatives of f(x, y, z) with respect to y and z respectively. 
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Example 3.2 


Solution 


For conciseness we sometimes use a suffix notation to denote partial derivatives, for 
example writing f. for Of/dx. The rules for partial differentiation are essentially the 
same as for ordinary differentiation, but it must always be remembered which variables 
are being held constant. 

Higher-order partial derivatives may be defined in a similar manner, with, for 
example, 


ax’ ax\ dx 
ta Bt 
ot. a( 2) «5, 
dzdyax dz\dyax) ~~ 


Find the first partial derivatives of the functions f(x, y, z) with formula (a) x + 2y + 2°, 
(b) x°(y + 2z) and (c) (x + y/(z? +). 


(a) f(x, y,z)=x+2y+z°*. To obtain f,, we differentiate f(x, y, z) with respect to x, 
keeping y and z constant. Thus f, = 1, since the derivative of a constant (2y + z°*) 
with respect to x is zero. Similarly, f, = 2 and f, = 32°. 


(b) f(x, y, z) =x°(y + 2z). Here we use the same idea: when we differentiate with 
respect to one variable, we treat the other two as constants. Thus 


od [x°(y + 2z)] =(y + 22) 2. (x?) = 2x(y + 2z) 
Ox Ox 

2 [x°(y + 2z)] aes (y + 2z) =x°(1) =x? 
oy oy 


a [x?(y + 2z)|= ge (y+ 2z)= x°(2) = 2x? 
oz Oz 


(c) fix, y, z) =(x + y)(z* + x). Here we use the same idea, together with basic rules 
from ordinary differentiation: 


Of — A(z’ +x) - (x+y)C) 


(quotient rule) 


ox (z +x) 
i oe, 
(z° + x) 
of __| 
Oy 4x 
2 
of _ =3z (ety) (chain rule) 
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The partial derivatives f, and f, of the function f(x, y), with respect to x and y respec- 
tively, are given by the commands 


MATLAB MAPLE 

syms xX y 

i = (se, W) = 8S Gs, 3) 8 

ee = Gib (iE, ss) ie g= Gbbr (i, Ss) 6 
iy = Caer, wy) iby ga Caer, Ws 


These commands can readily be extended to functions of more than two variables. 
Also second-order partial derivatives can be obtained by suitably differentiating the 
first-order partial derivatives already found. Thus in MATLAB the second-order 
partial derivatives of f(x, y) are given by 


yee S CUVEE (ibe, x) iey = chit (ise, y) yar = Chir (idy,%y) 
yee = GLEE (iby, =<) 


Alternatively, the non-mixed derivatives can be obtained directly using the 
commands 


ee =] CLE GE, <2) iyay =| hist Ge ,w,, 3) 


which can be extended to higher-order partial derivatives. The corresponding com- 
mands in MAPLE are 


ieee BS Cis (ie, sk, o%) f esa ge elalieie (Ej me sy) 6 


iyay g= Gul (it, a) 8 


In Example 3.2 we used the chain (or composite-function) rule of ordinary 
differentiation 


df _ df du 
dx dudx 


to obtain the partial derivative Of/dz. The multivariable calculus form of the chain rule 
is a little more complicated. If the variables u, v and w are defined in terms of x, y and 
z then the partial derivative of f(u, v, w) with respect to x is 


af _ af du, of dv , af dw 
Ox oOuodx Ovox dw ox 


with similar expressions for Of/dy and Of/dz. 


Find 07/or and OT/0@ when 
T(x, y) =x? —xyt+y? 
and 


x=rcos@ and y=rsin@ 
By the chain rule, 


ar _ ar ax , AT ay 
Or oxor oyor 
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In this example 


oT ax? y and ety? 
and 

Z = cose and 2 — sin 9 
so that 


a = (3x* — y)cos6 + (—x + 3y*)sin 6 
a 


Substituting for x and y in terms of r and @ gives 


oT _ 3r?(cos*@ + sin*@) — 2r cos @ sin 8 


or 
Similarly, 

or = 

oo 


= 3r°(sin 0 — cos 8)cos 6 sin@ + r7(sin’@ — cos’@) 


(3x? — y)(-rsin@) + (-x + 3y’)rcos@ 


Example 3.4 Find dH/dt when 
A(t) = sin(3x — y) 
and 


x=2-3 and y=}t?-5t+1 


Solution We note that x and y are functions of tf only, so that the chain rule becomes 


dH _ dH dx , OH dy 
dt oxdt oydt 


Note the mixture of partial and ordinary derivatives. H is a function of the one variable 
t, but its dependence is expressed through the two variables x and y. 
Substituting for the derivatives involved, we have 


aE = 3[cos(3x — y)]4t — [cos(3x — y)](¢— 5) 


= (11t+ 5)cos(3x — y) 
= (11¢+ 5)cos(#? + 5t— 10) 


Example 3.5 A scalar point function f(r) can be expressed in terms of rectangular cartesian coordin- 
ates (x, y, z) or in terms of spherical polar coordinates (r, 0, @), where 


x=rsinOcosé, y=rsin@sing, z=rcosd 
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as shown in Figure 3.4. Find Of/dx in terms of the partial derivatives of the function 
with respect to r, 0 and @. 


Solution Using the chain rule, we have 


of _ ofor , of 00, of mH 
ox or ox 00 ae Oo Ox 


From Figure 3.4, r° =x? + y? +z’, tan @ = y/x and tan@ = (x? + y*)!/z, so that 


or =*=sin@ cos 

ox or 

Ob _ 2( tan” ) _~__y_ __ sing 

Ox ox x x+y r sin @ 
Figure 3.4 Spherical 7 2, 21/2 
polar coordinates. 20 2) tan! |(@@ ty)” = aa NON TaET 

Ox ox Zz (x+y +z )(x 4+y) 

cos @ cos 8 
r 
Thus 
of sin @ cos ¢ 2 - Sn at cospe 
ry rsind ae r 00 


Example 3.6 — The Laplace equation in two dimensions is 
ax ay 

where x and y are rectangular cartesian coordinates. Show that expressed in polar co- 

ordinates (7, 0), where x =r cos 9 and y =r sin8, the Laplace equation may be written 


1 (rd) 4 1 Ju _ 
ror rae 


=0 


Solution Using the chain rule, we have 


du _ Quox dudy 
or aor yor 


Ou ou 


= aoe 38nd 
and 
Tu _ Fu og 0+ 2 sin? 94221 sin @ cos @ 
or ax” ay axdy 
Similarly 
Ou _ Our r sin a+ Mer cos 0) 
00 ax 
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and 
Ju = 2H. sin 9)” +2 y cos 0)° ~25 Fu rsin@ cos @ 
00 ay ax dy 
- ot cos 9) - 0 sin 0) 
so that 
Lu _ Fu i in?9 + 2 cos? 6-2 au sin 6 cos @ 
roe a oy’ ax dy 
ou ou 
- i Heos os O+5 "sin 6) 
Hence 
lou 4ldu _ Ju .2 Fu, au. 
= — 2 
ae -— mee sin 0+ — aye s @- x aa O cos 0 
and 
1 Ju 4ldu +ou a0 we ou 
r 06 *, ar ax’ ay” 
Since 
Zr ou) _ tas ou 
or\ or De ee 


we obtain the polar form of the Laplace equation in two dimensions 


12, ot) lou _ 
ror r oO 


The chain rule can be readily handled in both MATLAB and MAPLE. Considering 
Example 3.3, in MATLAB the solution may be developed as follows: 
The commands 


Swims 26 iy @ icleite 

Tase*3 = asgeye =: We Wise = cli (W@,s<) p Wye = Cait (a7) 9 

x S Ecos (icine) 9 sy S eesti (ielaSicel)) 2 

Sie SGU (Se, 7 )) ep SeciaSties, = Clair (sz, cinetea) » wae = GLEE (7,1) 9 
ytheta = diff(y,theta); 

Mie ey Ubercart nye 


return 


Wig = {(Sye OZ)  y)) catcloyen(iclaveiere))) =s {=e 25 Shui 2) )) eatetaiay (ela) 


To substitute for x and y in terms of r and theta we make use of the eval 
function, with 


eval (Tr) ; pretty(ans) 
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(3r*cos(theta)* — rsin(theta))\cos(theta) + 


(-rcos (theta) + 3r’sin(theta)’)sin(theta) 


which readily reduces to the answer given in the solution. 


Similarly the commands 


Ttheta = Tx*xtheta + Ty*ytheta; 
eval (Ttheta); pretty(ans) 


return the answer 


(-3r’cos (theta)? + rsin(theta))rsin(theta) + 


(-rcos (theta) + 3r’sin(theta)’)rcos (theta) 


which also reduces to the answer given in the solution. 
MAPLE solves this problem much more efficiently using the commands 


HE Oe (Ge WAS SS SS) yy cb gy shG 


Giff (T(r*cos(theta), 
ditt (@(e* eos (theta), 


Collec (3,1) 2 


returning the answer 


(-3cos (@)*sin(@) 


3.1.2 Exercises 


TeeSavin (ie laierce))) )), 16) 
r*sin(theta)), theta); 


+ 3sin(@)?cos(@))r 
(sin(@)? - cos(@)?)r? 


3 


Check your answers using MATLAB or MAPLE whenever possible. 


Sketch the contours (in two dimensions) of the 
scalar functions 


(a) f(x,y) =In@’ + y?- 1) 
(b) f(x, y) =tan'[y/(U +x)] 


Sketch the flow lines (in two dimensions) of the 
vector functions 


(a) F(x, y) =yi + (6x? — 4x)j 

(b) Fx, y) =yit Gx? - xj 

where i and j are unit vectors in the direction of 
the x and y axes respectively. 

Sketch the level surfaces of the functions 


(a) Sr) =Z—xy (b) f(r) =z-e” 


Sketch the field lines of the functions 
(a) F(r)=(@y, ¥ + 1, z) 
(b) F(r) = (yz, 2x, xy) 
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Find all the first and second partial derivatives of 
the functions 
(a) ff)=xyz-x?+y-z (0) f() =x"yz" 


(c) f(r) =z tan”'(y/x) 


Find df/dt, where 


(a) fr) =x? +y?-—z, andx=f- 1, y= 21, 
z= I(t-1) 


(b) f(r) =xyz, andx =esint, y=e“cost,z=t¢ 


Find of/oy and of/dz in terms of the partial 
derivatives of f with respect to spherical polar 
coordinates (r, 0, 6) (see Example 3.5). 


Show that if u(r) = f(r), where r? = x? + y* +z’, as 
usual, and 


“ed 
du, Iu, Fu 


2 2 7 = 9 
ox” Oye az 


192 


VECTOR CALCULUS 


then satisfies the differential equation 
dr’ ox oy az 


Hence find the general form for f(r). 


Show that 


10 _s~Verify that V(x, y, z) = sin3x cos4y cosh5z satisfies 
the differential equation 


1 Pda? IV IV, OV _ 
V(x, y, 2) = texp(-E 2) 5+ 5+55 =0 


ox oy az 


3.1.3 Transformations 


Example 3.3 may be viewed as an example of transformation of coordinates. For 
example, consider the transformation or mapping from the (x, y) plane to the (s, 4) 
plane defined by 


s=s(%,y),  t=4(x, y) (3.1) 
Then a function u = f(x, y) of x and y becomes a function u = F(s, t) of s and ¢ under the 
transformation, and the partial derivatives are related by 

du _ duds , du at 

ox odsdx odtoax 


(3.2) 
du _ duds , du at 
Oy oasdy odtoy 
In matrix notation this becomes 
au| [as at || au 
Ox _ ox ox|| das (3.3) 


du} jas dt) | du 

oy Oy oy)|| ot 
The determinant of the matrix of the transformation is called the Jacobian of the trans- 
formation defined by (3.1) and is abbreviated to 


As, or simply to J 
Hx, y) 
so that 


as dt| jas as 
eo As, t) _|Ox ax} jax ody (3.4) 
Tae) les ae een ar ; 


oy oy ox oy 


The matrix itself is referred to as the Jacobian matrix and is generally expressed in 


as as 
the form . : . The Jacobian plays an important role in various applications of 

t t 

ox oy 
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mathematics in engineering, particularly in implementing changes in variables in multi- 
ple integrals, as considered later in this chapter. 

As indicated earlier, (3.1) define a transformation of the (x, y) plane to the (s, ¢) plane 
and give the coordinates of a point in the (s, f) plane corresponding to a point in the 
(x, y) plane. If we solve (3.1) for x and y, we obtain 


x= X(s, 0), y= Ys, f) (3.5) 


which represent a transformation of the (s, f) plane into the (x, y) plane. This is called 
the inverse transformation of the transformation defined by (3.1), and, analogously to 
(3.2), we can relate the partial derivatives by 


du _ dude , udy 
Os oxds dyads 


du _ dude , dudy 
ot oxot odyot 


(3.6) 


The Jacobian of the inverse transformation (3.5) is 


_ Ax, y) _ Xs Vs 
i = 
As, t) xX, 


where the suffix notation has been used to denote the partial derivatives. Provided 
J #0, it is always true that J, = J7' or 


Ax, y) As, 1) _ 1 
As, t) Ax, y) 


If J= 0 then the variables s and ¢ defined by (3.1) are functionally dependent; that is, a 
relationship of the form f(s, f) = 0 exists. This implies a non-unique correspondence 
between points in the (x, y) and (s, ¢) planes. 


If s = s(x, y), = tx, y) then using MuPAD in MATLAB the commands 


delete x, y: 
limeless jacooiem((s, itll, se, wil) 


as as 
return the Jacobian matrix dx dy 
at at 
ox oy 


The same result may be obtained with the Symbolic Math Toolbox using the 
commands 


syms x y 6 E 
Jeccioigma (is, el, bs, wl) 


or in MAPLE using the commands 


with(VectorCalculus): 
Jacobian is, et], ke, wi)s 
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Example 3.7 (a) 


(b) 


Solution (a) 


(b) 


Obtain the Jacobian J of the transformation 
S=2x+y, t=x-—2y 


Determine the inverse transformation of the above transformation and obtain its 
Jacobian J,. Confirm that J, =J7. 


Using (3.4), the Jacobian of the transformation is 


y= Ge.0 21" a 
oy). lit. <2 


Solving the pair of equations in the transformation for x and y gives the inverse 
transformation as 


x= $Qs+2), y= i(s- 20 
The Jacobian of this inverse transformation is 


Re Ax, Y) _ 
‘As, t) 


2 
5 
ul 
5 


confirming that J; =J™'. 


Example 3.8 — Show that the variables x and y given by 


RS 5 


= it! (3.7) 
S t 


are functionally dependent, and obtain the relationship f(x, y) = 0. 


Solution The Jacobian of the transformation (3.7) is 


1 
Oey) t 
As, t) 


ft 

S 
Ke VY) 1 _s st st 
S 


2 


Since J = 0, the variables x and y are functionally related. 
Rearranging (3.7), we have 


xa1t, yaw] 


Ss 


so that 


(-Dy- steal 
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giving the functional relationship as 


xy-(x+y)=0 


The definition of a Jacobian is not restricted to functions of two variables, and it is 
readily extendable to functions of many variables. For example, for functions of three 
variables, if 


u= Ux, y, Z), v= Vix, ys Z), Wa W(x, y, Zz) (3.8) 


represents a transformation in three dimensions from the variables x, y, z to the variables 
u, Vv, w then the corresponding Jacobian is 


, DB, Ti 

Au, v, w) 
J= — =|, G wy, 
A(x, y, Z) : 


Again, if /= 0, it follows that there exists a functional relationship f(u, v, w) = 0 between 
the variables u, v and w defined by (3.8). 


3.1.4 Exercises 


Show that if x + y =u and y = wv, then ILS) 


Ax, y) _ 
O(u, v) 


Show that, ifx+y+z=u,y+z=uv andz=ww, 
then 


Ax, y,Z S52, iG 
Au, v, w) 


If x = e“ cosv and y = e" sinv, obtain the two 
Jacobians 


x, vy) an Au, v) 
Au, v) Ax, y) 


and verify that they are mutual inverses. 
é LZ 
Find the values of the constant parameter A for 
which the functions 
u=cosxcosy — Asinx siny 


v =sinxcosy + Acosxsiny 


are functionally dependent. 
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Find the value of the constant K for which 
u= Kx? + 4y*? +2? 
v=3x+2y+z 
w = 2yz + 3zx + Oxy 

are functionally related, and obtain the 


corresponding relation. 


Show that, if w = g(x, y) and v = A(x, y), then 
ori fp oe ly 


Qu oy ov oy 
dy __dv dy _ du 
Ou Ox [i Ov ox fy 
where in each case 
J= Au, v) 
Ax, y) 


Use the results of Exercise 16 to obtain the partial 
derivatives 


ou ov au ov 
where 


u=e'cosy and v=e™“siny 
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SL 


Figure 3.5 
Illustration of result 
(3.11). 


The total differential 


Consider a function u = f(x, vy) of two variables x and y. Let Ax and Ay be increments 
in the values of x and y. Then the corresponding increment in w is given by 


Au = f(x + Ax, v + Ay) — f(x, y) 


We rewrite this as two terms: one showing the change in wu due to the change in x, and 
the other showing the change in w due to the change in y. Thus 


Au= [fx ae Ax, yt Ay) —f(x, yt Ay)] 1. Lf, yt Ay) —f(x, y)] 
Dividing the first bracketed term by Ax and the second by Ay gives 


fom xt Ax, yt Ay) -~ fx, y+ Ax) ,,. 4: f(x, y+ Ay) - fx, Yay 
Ax Ay 
From the definition of the partial derivative, we may approximate this expression by 
Au = rx ar Gis 
Ox oy 
We define the differential du by the equation 
du = of ax + Ory (3.9) 
ox oy 
By setting f(x, y) =/f\(, y) =x and f(x, vy) = fA@, y) = y in turn in (3.9), we see that 


d 0 


so that for the independent variables increments and differentials are equal. For the 
dependent variable we have 


dx = Tiny + Hi ny =Ax and dy=Ay 
x Vv 


dws ales + any (3.10) 
ox oy 


We see that the differential dw is an approximation to the change Aw in u = f(x, y) 
resulting from small changes Ax and Ay in the independent variables x and y; that is, 


Au = du= TE dee of ay = Saxe Ca (3.11) 
Ox oy ox oy 


a result illustrated in Figure 3.5. 


(x + Ax, y + Ay, u+ Au) 
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This extends to functions of as many variables as we please, provided that the partial 
derivatives exist. For example, for a function of three variables (x, y, z) defined by 
u = f(x, y, z) we have 


Kye SNe OL Oley 
ox oy oz 
Sel i ah 

aoe ee oe 


The differential of a function of several variables is often called a total differential, 
emphasizing that it shows the variation of the function with respect to small changes in 
all the independent variables. 


Find the total differential of u(x, y) = x’. 


Taking partial derivatives we have 


Ou _ ot 


ox 
Hence, using (3.10), 
du = yx’! dx + x” Inx dy 


Differentials sometimes arise naturally when modelling practical problems. When this 
occurs, it is often possible to analyse the problem further by testing to see if the expres- 
sion in which the differentials occur is a total differential. Consider the equation 


P(x, y) dx + O(x, y) dv = 0 


connecting x, vy and their differentials. The left-hand side of this equation is said to be 
an exact differential if there is a function f(x, y) such that 


df= P(x, y) dx + O(x, y)dy 


Now we know that 
df= Sl dey Sl ay 
ox oy 
so if f(x, y) exists then 
Po.yy=Z and O(x,=£ 
ox oy 


For functions with continuous second derivatives we have 


ia 
Oxdy ayax 
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Example 3.10 


Solution 


Thus if f(x, y) exists then 


ap _ a0 2 
oy ox ares 


This gives us a test for the existence of f(x, y), but does not tell us how to find 
it! The technique for finding f(x, v) is shown in Example 3.10. 


Show that 
(6x + 9y + 11)dx + (9x — 4y + 3)dy 
is an exact differential and find the relationship between y and x given 


dy _ _6x+9y+11 
dx 9x-4y4+3 


and the condition y = | when x = 0. 


In this example 
P(x, y)=6x+9y+11 and O(x, y)=9x-4y4+3 
First we test whether the expression is an exact differential. In this example 


dP 
oy 


so from (3.12), we have an exact differential. Thus we know that there is a function 


=9 and Og’ 5 
Ox 


F(x, y) such that 


A GeaOye gad Chg aes (3.13a, b) 
ox oy 


Integrating (3.13a) with respect to x, keeping y constant (that is, reversing the partial 
differentiation process), we have 


f(x, y) = 3x* + Oxy + 11x + g(y) (3.14) 


Note that the ‘constant’ of integration is a function of y. You can check that this expression 
for f(x, y) is correct by differentiating it partially with respect to x. But we also know 
from (3.13b) the partial derivative of f(x, y) with respect to y, and this enables us to find 
g’(v). Differentiating (3.14) partially with respect to y and equating it to (3.13b), we have 


Of - 9x 498 ~ 9x 4y 43 
oy dy 


(Note that since g is a function of y only we use dg/dy rather than 0g/oy.) Thus 


dg _ 
dy 


SO, on integrating, 


—4y+3 


a(y) =—2y? + 3y+C 
Substituting back into (3.13b) gives 
S(x,y) = 3x? + Oxy + 11x — 2y? + 3y +C 
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Now we are given that 


dy 6x +9y+11 


dx 9x-4y4+3 
which implies that 

(6x + 9y + 11)dx + (9x — 4y + 3)dv=0 
which in turn implies that 

3x* + Oxy + Llx -— 2y? + 3y+C=0 


The arbitrary constant C is fixed by applying the given condition y = 1 when x = 0, 
giving C=—1. Thus x and y satisfy the equation 


3x? + Oxy + llx — 2y? + 3y=1 


3.1.6 Exercises 


18 Determine which of the following are exact is the exact differential of a function f(x, y). Find the 
differentials of a function, and find, where corresponding function f(x, y) that also satisfies the 
appropriate, the corresponding function. condition f(0, 1) = 0. 

(a) (y? + 2xy + 1)dx + (2xy + x’) dy 20 Show that the differential 
(b) (2xy? + 3y cos 3x) dx + (2x’y + sin 3x) dy g(x, y) = (10x? + 6xy + 6y*) dx 


+ (9x° + 4xy + 15y*) dy 


(c) (6xy — y*) dx + (2x e” — x”) dy 


is not exact, but that a constant m can be chosen so 


(d) (2? — 3y) dx + (12y? — 3x) dy + 3xz7dz that 


(2x + 3y)"g(x, y) 


19 Find the value of the constant A such that 


is equal to dz, the exact differential of a function 


(ycosx + Acosy) dx + (xsiny + sinx + y) dy z=f(x, y). Find f(x, y). 


Derivatives of a scalar point function 


3.2.1 


In many practical problems it is necessary to measure the rate of change of a scalar 
point function. For example, in heat transfer problems we need to know the rate of 
change of temperature from point to point, because that determines the rate at which 
heat flows. Similarly, if we are investigating the electric field due to static charges, 
we need to know the variation of the electric potential from point to point. To deter- 
mine such information, the ideas of calculus were extended to vector quantities. The 
first development of this was the concept of the gradient of a scalar point function. 


The gradient of a scalar point function 


We described in Section 3.1.1 how the gradient of a scalar field depended on the direc- 
tion along which its rate of change was measured. We now explore this idea further. 
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Consider the rate of change of the function f(r) at the point (x, y, z) in the direction of 
the unit vector (/, m, n). To find this, we need to evaluate the limit 


iit Art an An 


Ar>0 
where Ar is in the direction of (/, m, n). In terms of coordinates, this means 


r+Ar=r+Ar(i, m,n) 
= (x + Ax, y t+ Ay, z+ Az) 


so that 
Ax = IAr, Ay = mAr, Az =nAr 


Thus we have to consider the limit 
my Leet Ar, mar z+nAr) - f(x, y,Z 
ae 
We can rewrite this as 


lim 
Ar>0 


a fies a ytmAr, z+nAr) - f(x, y, alt 


Ar30 mAr 


jets +mAr, z+nAr) - f(x, y+mAr senan| 
[Ar 


5 Vii a y, z+nAr) - f(x, y, ah 


Ars0 nAr 


Evaluating the limits, remembering that Ax = /Ar and so on, we find that the rate of 
change of f(r) in the direction of the unit vector (/, m, n) is 
i+ Lm + or =(2 of a. ] 
oy woe ox’ dy az ett) 
The vector 
(Z of of } 
ox’ oy’ oz 
is called the gradient of the scalar point function f(x, y, z), and is denoted by grad f or 
by Vf, where V is the vector operator 
.0 ,.0 0 
V=i-+j—t+k— 
he tap ae 


where i, j and k are the usual triad of unit vectors. 
The symbol V is called ‘del’ or sometimes ‘nabla’. Then 


yy = ye a) cpa el oh var 
Bee. Deore & oy’ = c=) 


Thus we can calculate the rate of change of f(x, y, z) along any direction we please. If 
i is the unit vector in that direction then 


(grad f) a 
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(a) Adjacent level 
surfaces of f(r); 
(b) grad facts 
normally to the 
surface f(r) = c. 


Example 3.11 


Solution 
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gives the required directional derivative, that is the rate of change of f(x, y, z) in the 
direction of #. Remembering that a-b = |a||b|cos@, where @ is the angle between the 
two vectors, it follows that the rate of change of f(x, y, z) is zero along directions per- 
pendicular to grad fand is maximum along the direction parallel to grad f Furthermore, 
grad f acts along the normal direction to the level surface of f(x, y, z). We can see this 
by considering the level surfaces of the function corresponding to c and c + Ac, as 
shown in Figure 3.6(a). In going from P on the surface f(r) = c to any point Q on 
f(r) =c + Ac, the increase in fis the same whatever point Q is chosen, but the distance 
PQ will be smallest, and hence the rate of change of f(x, vy, z) greatest, when Q lies on the 
normal fi to the surface at P. Thus grad fat P is in the direction of the outward normal 
fi to the surface f(r) = u, and represents in magnitude and direction the greatest rate of 
increase of f(x, y, z) with distance (Figure 3.6(b)). It is frequently written as 


grad f = H 


where Of/On is referred to as the normal derivative to the surface f(r) = c. 


f(r=ct+Ac 


f(r)=e 


Find grad f for f(r) = 3x? + 2y* + z* at the point (1, 2, 3). Hence calculate 


(a) the directional derivative of f(r) at (1, 2, 3) in the direction of the unit vector 
1 
3 (2, 2, 1); 
3 


(b) the maximum rate of change of the function at (1, 2, 3) and its direction. 


(a) Since df/dx = 6x, Of/dy = 4y and Of/dz = 2z, we have from (3.15) that 
grad f= Vf= 6xi + 4yj + 2zk 
At the point (1, 2, 3) 
grad f= 6i + 8j + 6k 


Thus the directional derivative of f(r) at (1, 2, 3) in the direction of the unit vector 
G53) is 
(6i+ 87+ 6k): (2i+2f+tk)=% 

(b) The maximum rate of change of f(r) at (1, 2, 3) occurs along the direction parallel 
to grad fat (1, 2, 3); that is, parallel to (6, 8, 6). The unit vector in that direction 
is (3, 4, 3)//34 and the maximum rate of change of f(r) is | grad f| = 2/34. 
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Example 3.12 


Solution 


Figure 3.7 Tangent 
plane at (1, 3, 5) to the 
paraboloid 2z = x* + y”. 


If a surface in three dimensions is specified by the equation f(x, y, z) = c, or equival- 
ently f(r) = c, then grad fis a vector perpendicular to that surface. This enables us to 
calculate the normal vector at any point on the surface, and consequently to find the 
equation of the tangent plane at that point. 


A paraboloid of revolution has equation 2z = x + y*. Find the unit normal vector to the 
surface at the point (1, 3, 5). Hence obtain the equation of the normal and the tangent 
plane to the surface at that point. 


A vector normal to the surface 2z = x* + y’ is given by 

grad (x? + y? — 2z) = 2xi + 2yj — 2k 
At the point (1, 3, 5) the vector has the value 2i + 6j — 2k. Thus the normal unit vector 
at the point (1, 3, 5) is (+ 37 — k)//11. The equation of the line through (1, 3, 5) in the 
direction of this normal is 


fe) fo) 4 
1 3 =| 


and the equation of the tangent plane is 


(D@- 1+ B)(y- 3) + CDE-5)=90 
which simplifies to x + 3y — z= 5 (see Figure 3.7). 


The concept of the gradient of a scalar field occurs in many applications. The 
simplest, perhaps, is when /(r) represents the potential in an electric field due to static 
charges. Then the electric force is in the direction of the greatest decrease of the poten- 
tial. Its magnitude is equal to that rate of decrease, so that the force is given by —gradf. 


Using the Symbolic Math Toolbox in MATLAB the gradient grad f of the scalar func- 
tion f(x, y, z) is given by the grad function. For example, considering Example 3.11, 
the gradient of the scalar function f(x, y, z) = 3x° + 2y’ +z is given by the commands 


syms x y Z 
f= (Boer 2 2 BiyoR 2 we D)) g 

Geechee = [Chiaki (ie ,s<) , Chie (ie 57), tlie (iE, 74) || 8 
pretty (gradf) 


returning the answer 
[6x 4y 22] 
Using MuPAD the answer is returned using the commands 


delete x, y, 2Z: 
Isinaile ge qmach(Sex*2 2 2y2 22°2, (sx, Ww, ll) 


In MAPLE the answer is obtained using the commands 


with(VectorCalculus) : 
gieewhe s= Girachiemc (Seo + Beye & 22, [se, Ww, 21) 5 
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3.2.2 Exercises 


Find grad f for f(r) = x’yz* at the point (1, 2, 3). 
Hence calculate 


(a) the directional derivative of f(r) at (1, 2, 3) 
in the direction of the vector (—2, 3, —6); 

(b) the maximum rate of change of the function at 
(1, 2, 3) and its direction. 

Find Vf where f(r) is 

(b) ztan (y/x) 

(c) ers I(x? +y’) 

(d) xyzsin {u(x + y +z)} 


(a) x°+y?-z 


Find the directional derivative of f(r) = x* +? —z 
at the point (1, 1, 2) in the direction of the vector 
(4, 4, —2). 


Find a unit normal to the surface xy? — 3xz =—5 at 
the point (1, —2, 3). 


Ifr is the usual position vector r= xi + yj + zk, with 
|r| =r, evaluate 


(a) Vr (b) v(t} 


If Vo= (2xy +: 2°)i + (x° +: 2)j + (y + 2xz)k, find a 
possible value for @. 


Given the scalar function of position 

O(x, y, Z) =x°y — 3xyzt23 
find the value of grad @ at the point (3, 1, 2). Also 
find the directional derivative of @ at this point in 


the direction of the vector (3, —2, 6); that is, in the 
direction 3i — 2j + 6k. 


28 


29 


30 


Find the angle between the surfaces x7 + y?+2°=9 
and z = x? + y? — 3 at the point (2, —-1, 2). 


Find the equations of the tangent plane and normal 
line to the surfaces 

(a) x°+2y?+3z*=6at (1, 1, 1) 

(b) 2x*+y?-z? =-3 at (1, 2, 3) 


(c) x°+y?-z=1at(l, 2, 4). 


(Spherical polar coordinates) When a function f(r) 
is specified in polar coordinates, it is usual to 
express grad fin terms of the partial derivatives of f 
with respect to r, @and @ and the unit vectors u,, Ug 
and w, in the directions of increasing r, @ and @ as 
shown in Figure 3.8. Working from first principles, 
show that 


= =F, gt |_ of 
Vi= grads ore i a6°8 r sin 006 ue 


Figure 3.8 Unit vectors associated with spherical 
polar coordinates. 


Derivatives of a vector point function 


When we come to consider the rate of change of a vector point function F(r), we see 
that there are two ways of combining the vector operator V with the vector F. Thus we 
have two cases to consider, namely 


V:-F and VxF 


that is, the scalar product and vector product respectively. Both of these ‘derivatives’ 
have physical meanings, as we shall discover in the following sections. Roughly, if we 
picture a vector field as a fluid flow then at every point in the flow we need to measure 


www.2Ofile.org 


204 VECTOR CALCULUS 


ce | 


Figure 3.9 Flow out 
of a cuboid. 


the rate at which the field is flowing away from that point and also the amount of spin 
possessed by the particles of the fluid at that point. The two ‘derivatives’ given formally 
above provide these measures. 


Divergence of a vector field 


Consider the steady motion of a fluid in a region R such that a particle of fluid instan- 
taneously at the point r with coordinates (x, y, z) has a velocity v(r) that is independent 
of time. To measure the flow away from this point in the fluid, we surround the point 
by an ‘elementary’ cuboid of side (2Ax) x (2Ay) x (2Az), as shown in Figure 3.9, and 
calculate the average flow out of the cuboid per unit volume. 


k- v(x, y,z+ Az) 


i. v(x — Ax, y, z) 


i- v(x + Ax, y, Z) 


jv, y + Ay, z) 
k - v(x, y, z— Az) 


The flow out of the cuboid is the sum of the flows across each of its six faces. 
Representing the velocity of the fluid at (x, y, z) by v, the flow out of the face ABCD is 
given approximately by 


i-v(x + Ax, y, z)(4AyAz) 
The flow out of the face A’B’C’D’ is given approximately by 
—i- v(x — Ax, y, z)(4AyAz) 


There are similar expressions for the remaining four faces of the cuboid, so that the total 
flow out of the latter is 


i: [v(x + Ax, y, z) — v(x — Ax, y, z)](4AvAz) 
+7: [v@, y + Ay, z) — v(x, y — Ay, z)](4AxAz) 
+k: [v@, y, z+ Az) — v@, y, z— Az)](4AxAy) 


Dividing by the volume 8AxAyAz, and proceeding to the limit as Ax, Ay, Az > 0, we 
see that the flow away from the point (x, y, z) per unit time is given by 
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This may be rewritten as 


.0 .,0 é 
i—+j—t+k—|-v 
( ox In oz 
or simply as V-v. Thus we see that the flow away from this point is given by the scalar 
product of the vector operator V with the velocity vector v. This is called the divergence 
of the vector v, and is written as divv. In terms of components, 


dives G3 eee x2) (ees) 
Boy , doy , dvs 
See 


(3.16) 


When v is specified in this way, it is easy to compute its divergence. Note that the 
divergence of a vector field is a scalar quantity. 


Find the divergence of the vector v = (2x — y*, 3z + x”, 4y — z’) at the point (1, 2, 3). 


Here v, = 2x — y”, vy) =3z+ x? and v, = 4y — 2’, so that 


dy By _ Be 


= —2 
ox > oy , Oz * 


Thus from (3.16), at a general point (x, y, z), 
divv=V-v=2-2z 

so that at the point (1, 2, 3) 
V-v=-4 


A more general way of defining the divergence of a vector field F(r) at the point r 
is to enclose the point in an elementary volume AV and find the flow or flux out of AV 
per unit volume. Thus 


P= ae = ep 
AV-0 AV 


A non-zero divergence at a point in a fluid measures the rate, per unit volume, at which 
the fluid is flowing away from or towards that point. That implies that either the density 
of the fluid is changing at the point or there is a source or sink of fluid there. In the case 
of a non-material vector field, for example temperature gradient in heat transfer, a non- 
zero divergence indicates a point of generation or absorption. When the divergence is 
everywhere zero, the flow entering any element of the space is exactly balanced by the 
outflow. This implies that the lines of flow of the field F(r) where div F = 0 must either 
form closed curves or finish at boundaries or extend to infinity. Vectors satisfying this 
condition are sometimes termed solenoidal. 
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Sil 


32 


SS) 


34 


65 


(2) Using MuPAD in MATLAB the divergence of a vector field is given by the 
divergence function. For example, the divergence of the vector 


v=(2x-y, 3z +x, 4y—-7) 


considered in Example 3.13, is given by the commands 


delete x, y, 2Z: 


linalg 


which return the answer 
2 = 22 


divergenc 


Bo a ON AA > SR || 


In MAPLE the answer is returned using the commands 


with(VectorCalculus) : 


SetCoordinates(‘cartesian’ 
F:= VectorPield(<2*x -y~2, 


Divergence(F); or Del.F 
3.3.2 Exercises 
Find div v where 
(a) v(r) = 3x°yit i +x7k 
(b) v(r) = (3x+y)it (224+ x)j + (2 - 2y)k 36 


If F = (2xy? +.27)i + (3x22? — 723) + (yz? —xz°)k, 
calculate div fat the point (—1, 2, 3). 


Find V(a-r), (a: V)r and a(V-r), where a is a 
constant vector and, as usual, ris the position vector 
r= (x, y, Z). 


The vector v is defined by v = rr7', where 
r= (x, y, z) andr =|r|. Show that 


V(V -v) = grad divv=-42r 


I 


Find the value of the constant A such that the vector 
field defined by 


3.3.3 Curl of a vector field 


Sy 


[ 8o We @ll)s 
Be AD Ay = See Ds) g 


, 


F = (2x°y? +. 2°)i+ (3xy? — x°z)j + (Axy?z + xy)k 
is solenoidal. 
(Spherical polar coordinates) Using the notation 


introduced in Exercise 30, show, working from first 
principles, that 


: 1d,2 ll a) : 
V-v=divv= = —(rv,) +—— —(y% sin 8 
fe Ganee 
1 oa 
+ — 
pemoon 


where v = v,.U, + Vollg + Ugly. 
A force field F,, defined by the inverse square law, 
is given by 
Fe=rir* 
Show that V- F = 0. 


It is clear from observations (for example, by watching the movements of marked corks 
on water) that many fluid flows involve rotational motion of the fluid particles. Com- 
plete determination of this motion requires knowledge of the axis of rotation, the rate 
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Figure 3.10 Flow 
around a rectangle. 
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of rotation and its sense (clockwise or anticlockwise). The measure of rotation is thus a 
vector quantity, which we shall find by calculating its x, y and z components separately. 
Consider the vector field u(r). To find the flow around an axis in the x direction at the 
point r, we take an elementary rectangle surrounding r perpendicular to the x direction, 
as shown in Figure 3.10. 

To measure the circulation around the point r about an axis parallel to the x direc- 
tion, we calculate the flow around the elementary rectangle ABCD and divide by its 
area, giving 


[vx(x, y*, z — Az)(2Ay) + v,(x, y + Ay, 2*)(2Az) 


— v,(x, §, z+ Az)(2Ay) — v3(x, y — Ay, Z)(2Az)]/(4AyAz) 


where y*, f €(y — Ay, y + Ay), 2*, Z €(z — Az, z+ Az) and v = vi + vj + v3k. 
Rearranging, we obtain 


—[v(x, ¥, z+ Az) — v,(x, y*, z — Az)|/(2Az) 
pa [v3(x, y + Ay, z*) ~ v;(x, yY— Ay, Z)\/(2Ay) 


Proceeding to the limit as AvAz > 0, we obtain the x component of this vector as 


dv; _ v2 
Oy az 

By similar arguments, we obtain the y and z components as 
Ov, Ov; Ov, ov, 


Oz Ox’ ox oy 


respectively. 
The vector measuring the rotation about a point in the fluid is called the curl 
of v: 


curl v = (2s - 2)j4 (Sr- Ba)j4 (e-Bay 


dy az dz ax ox oy 
It may be written formally as 
PME 
curl » =| 2 - g (3.18) 


i oe 
or more compactly as 


Cuidl p= W XH 
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Example 3.14 


Solution 


AS fi 


Q 


Figure 3.11 
Circulation around 
the element AS. 


Figure 3.12 
Rotation of a 
rigid body. 


Find the curl of the vector v = (2x — y*, 3z +. x”, 4y — z’) at the point (1, 2, 3). 


Here v, = 2x — y*, v) = 3z + x’, v; = 4y — 2’, so that 


i J k 
od o o 
ox oy Oz 

2x-y 3z4+x° 4y-z 


curl v = 


2 


See 2| 
[2a z) Aerts) 


ia -2)- S02x ¥) 


+ K[RGete) - $(2x J 


= i(4— 3) -j(O— 0) + A(2x + 2y) =i + 2Axt+y)k 
Thus, at the point (1, 2, 3), V x v= (1, 0, 6). 


More generally, the component of the curl of a vector field F(r) in the direction of the 
unit vector fi at a point L is found by enclosing L by an elementary area AS that is perpen- 
dicular to 7, as in Figure 3.11, and calculating the flow around AS per unit area. Thus 


¥ . flow round AS 
curl F)- 4 = lim ——————— 
( ) rar AS 


Another way of visualizing the meaning of the curl of a vector is to consider the 
motion of a rigid body. We can describe such motion by specifying the angular velocity 
o of the body about an axis OA, where O is a fixed point in the body, together with the 
translational (linear) velocity v of O itself. Then at any point P in the body the velocity 
u is given by 


u=v+@xr 
as shown in Figure 3.12. Here v and @ are independent of (x, y, z). Thus 


curl w= curlv + curl (@ x r) = 0 + curl (@ x r) 


A 
Ne Xr tangential velocity 


oO f : 
v_ translation velocity 
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The vector @ Xx r is given by 
@ X r=(@,, @, 3) X (x, y, Z) 
= (O22 — Oyy)i + (O3X — @2)j + (My — Oxx)k 


and 
i j k 
0 0 O 
l(@ xr) = =— = = 
ae, ox oy oz 
@5Z-O,V @3X-Q@,Z Wy - @x 
=2Q@,i+ 20,j+ 20;k=2@ 
Thus 
curl u = 2@ 
that is, 
@=tcurlu 


Hence when any rigid body is in motion, the curl of its linear velocity at any point is 
twice its angular velocity in magnitude and has the same direction. 

Applying this result to the motion of a fluid, we can see by regarding particles of the 
fluid as miniature bodies that when the curl of the velocity is zero there is no rotation 
of the particle, and the motion is said to be curl!-free or irrotational. When the curl is 
non-zero, the motion is rotational. 


Using MuPAD in MATLAB the command linalg :: curl(v, x) computes the 
curl of the three-dimensional vector field v with respect to the three-dimensional 
vector x in cartesian coordinates. For example, the curl of the vector 


v=(2x—-y’, 3z +x’, 4y—2) 
considered in Example 3.14, is given by the commands 


GIES oe, WW, 2S 


iinimevle; ¢2 eur (exe SR, Bem Boe aay = 2), 
ey Wy wll) 
1 
which return the answer 0 
25 32 Dy) 


In MAPLE the answer is returned using the commands 
with (VectorCalculus) : 
SetCoordinates(‘cartesian’ [ x, y, 2]); 
IPG) Wisteie@relsaellel(—awse Sy 2, Bier, se Se dba eA DIS) e 
(tae (inp Or well se iy 
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3.3.4 Exercises 


38 Find w= curly when v = (3xz’, —yz, x + 22). 43. Ifv=—yitxj+xyzk is the velocity vector of a fluid, 
find the local value of the angular velocity at the 


39 A vector field is defined by v = (yz, xz, xy). Show point (1, 3, 2). 


that curly = 0. ‘ 
44 Ifthe velocity of a fluid at the point (x, y, z) is given by 


40 Show that if v = (2x + yz, 2y + zx, 2z + xy) then v= (ax + by)i + (cx + dy)j 
curly = 0, and find f(r) such that v = grad/- find the conditions on the constants a, b, c and d in 
order that 
41 By evaluating each term separately, verify the ; 
div v=0, curlv = 0 


identity 
rify that in thi 
VX =AV xd) EW XY Verify that in this case 
v=} i grad (ax? + 2bxy — ay”) 
for f(r) = x* — y and u(r) = (z, 0, -x). 
45 (Spherical polar coordinates) Using the notation 
42 Find constants a, b and c such that the vector field introduced in Exercise 30, show that 


defined by Vxve=culv 


F = (4xy + az*)i + (bx? + 3z)j + (6xz7 + cy)k u rug sini, 


is irrotational. With these values of a, b and c, = 1 oa ao @ 
determine a scalar function (x, y, z) such that rsin@|or 0 ag 
F=V6¢. VD, Ve Tr Sin vy 


3.3.5 Further properties of the vector operator V 


So far we have used the vector operator in three ways: 


Vf = grad f = L; + a a Lg, f(r) a scalar field 
Vreven — on poe, Hs F(r) a vector field 
ox ee oz’ 
VxF=culF 
of; - 2); +(Z- sh); (2- ahs, 
-(¢ eee “ES J+ ae oF F(r) a vector field 


A further application is in determining the directional derivative of a vector field: 


aVF =(a,5+a5 +05 )F 


Ox 0 Oz 
Eh oh AH, ah); (a aH, AH, 2h); 
=(agitagt oo hee ee 
FY ees ae 
+ (ager ade +a \k 
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The ordinary rules of differentiation carry over to this vector differential operator, but 
they have to be applied with care, using the rules of vector algebra. For non-orthogonal 
coordinate systems a specialist textbook should be consulted. Thus for scalar fields f(r), 
g(r) and vector fields u(r), v(r) we have 


VifeOnl= 2VE (3.19a) 

VI SMa] = SOVAr) +fOVEr) (3.19b) 
V[u(r)-v(r)] =v x (V xu) +ux(Vxv)+(0-Vou+(u- Vv (3.19c) 
V-[f@yu(r)] =u-Vf+fV-u (3.19d) 
Vx [nun] = (Vf) xutfV xu (3.19e) 
V -[u(r) X v(r)] =v: (V xX u)-—u-(V Xv) (3.19f) 
V x [u(r) x v(r)] = (v- V)u — 0(V uu) — (u- Vv + u(V -v) (3.19g) 


Higher-order derivatives can also be formed, giving the following: 


div [grad f(r)] = V-Vf= of + Chl + oe = NE (3.20) 
ox Oy a 


where V? is called the Laplacian operator (sometimes denoted by A); 


curl [grad f(r)] = V x V(r) = 0 (3.21) 
since 
af OF OF pf OP OP af OF oor 
a aa Ge a, . Ge a r a oa) 
=0 


when all second-order derivatives of f(r) are continuous; 
div[curl v(r)] = V-(V x v) =0 (3.22) 
since 


2B 9) 2a 0) OO) 


Ox\ dy dz) dy\dz ox) dz\ ax oy 
v0) = 100-0) = (12452 442) Be 2) 
grad (div v) = V(V-v) = (+2 as + w2\ a + 5 ap > (3.23) 
2 SE as a eee 
Vv sere ee So Jott ras tok) (3.24) 
curl [curl v(r)] = V x (V x v) = V(V-v) — Vv (3.25) 
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Example 3.15 ‘Verify that V x (V x v) = V(V-v) — Vv for the vector field v = (3xz’, —yz, x + 22). 


i J k 
, 0 d d 
lut V =||—R = = |= 6xz- 1,0 
Solution Xv > ay a (y, 6xz ) 
3xz° —yz x+2z 
i j k 
a Q Q 
Vx(Vv =|> = =| = (-6x, 0, 6z- 1 
oe ox Oy az ae ee 
y 6xz-1 0 


pe 2 faxePy an SK oO re 
Vv a Ce ae yz) + S (e+ 22) = 32-242 


V(V -v) = (0, 0, 6z — 1) 
V°v = (V°(3xz’), V*(-yz), V*(x + 22)) = (6x, 0, 0) 


Thus 
V(V -v) — Vv = (—6x, 0, 6z- 1)=Vx(V xv) 


Similar verifications for other identities are suggested in Exercises 3.3.6. 


Example 3.16 = Maxwell’s equations in free space may be written, in Gaussian units, as 
(a) divH=0, (b) divE=0 


(c) curlW=VxH=12E, (d) mipev eRe lee 
c Ot c ot 
where c is the velocity of light (assumed constant). Show that these equations are 
satisfied by 
H=12 grad 6 x k, po 2 gary 
c Ot c Oot az 
where @ satisfies 
ven 338 
Cc Ot 


and k is a unit vector along the z axis. 


Solution (a) H=12 graddxk 
Cc 
gives 
div H= ie div (grad @ x k) 
c Ot 
=e ¢ [K- curl (grad @) — (grad ¢)-curlk], from (3.19f) 
c 


By (3.21), curl (grad @) = 0, and since k is a constant vector, curl k = 0, so that 
divH=0 
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ingest aiv( 422) adie 6 
c ot? az 


- -12(28),2 V0), by (3.20 
caz\or a ve 
- al 2 28) 
Sei Voges 
oz . c or 
and since V7 = (1/c’)0’¢/dt*, we have 
divE=0 


(c) curlH= Le curl (grad @ x k) 
c Ot 


= Bey: 
rae, [(A-V) grad 

— k (div grad @) — (grad @-V)k + grad @(V-A)], from (3.19g) 
7 ; s(2 grad @- KV") , since k is a constant vector 
_10E 

c ot 


(d) curlE = sour («28) + “ curl grad @ 
Cc ot re) 


eae oy map since curl grad = 0 by (3.21) 
ot 


- -1(.26,_,28.) 
Ovot TaxaP 


OH_ 1a 
— = -—grad Ox k 
re , 


= (grad @xk), since k is a constant vector 
co 
ee 3 |(dee+ 2+ S8n) xa = LF (08,00) 1 (26. _ ;.26 ) 
cot |\dx dy a cot \ dy a c 


so that we have 


10H 
VxE=--= 
. c ot 
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46 


47 


48 


49 


50 


Bl 


Topics in integration 


3.3.6 Exercises 
Show that if g is a function of r = (x, y, z) then 


grad g= 1dg, 
rdr 


Deduce that if u is a vector field then 
div [(u x r)g] = (r- curl u)g 

For O(x, y, z) =x°y’z? and 

F(x, y, Z) =x°yi + xy7zj — yz*k determine 


(a) Vd = (b) graddivF = (c) curlcurl F 


Show that if a is a constant vector and r is the 
position vector r = (x, y, z) then 


div {grad [(r-r)(r-a)]} = 10(r- a) 
Verify the identity 


V’v = grad div v — curl curl v 


for the vector field v = xy(xi + yj + Zk). 


Verify, by calculating each term separately, 
the identities 


div (u Xv) =v-curlu —u-curlv 


curl(u Xv) =udivv —vdivu+(v-V)u 
—(u-V)v 


when uw = xyj + xzk and v = xyi + yzk. 


If r is the usual position vector r = (x, y, Z), 
show that 


(a) div graa( } =0 


(b) curl c x w(!) + grad c : graa{ + } =0 


52 


BS) 


54 


55) 


If A is a constant vector and r is the position vector 
r = (x, y, z), show that 


(a) grad( 43") = as git); 


(b) curl (4s } = 24 


+3 (4x1) xr 
; 5 


1 
If r is the position vector r = (x, y, z), and a and b 
are constant vectors, show that 

(a) Vxr=0 

(b) (a:V)r=a 

(c) Vx [(a-r)b — (b- r)a] = 2(a x b) 

(d) V- [(a-r)b-(b- ra] =0 

By evaluating V-(V/), show that the Laplacian 


in spherical polar coordinates (see Exercise 30) is 
given by 
vip= 42 (2), 12 ino ) 
e or? r sin @00 00 
1_ of 


r sin’ 006 


Show that Maxwell’s equations in free space, namely 


div H = 0, divE =0 
vxHal Ze vxE=-loH 
c ot c ot 
are satisfied by 
H= 1 oun % 
c ot 


E=curlcurlZ 


where the Hertzian vector Z satisfies 
vz-t az 
c ot 


In the previous sections we saw how the idea of the differentiation of a function of a 
single variable is generalized to include scalar and vector point functions. We now turn 
to the inverse process of integration. The fundamental idea of an integral is that of 
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Figure 3.13 Definite 
integral as an area. 


3.4.1 


Figure 3.14 Integral 
along a curve. 
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summing all the constituent parts that make a whole. More formally, we define the 
integral of a function f(x) by 


b n 
| f(x)dx= tim SY) AR) Ax; 
n>0 4 
¢ all Ax, 30 =! 
where d =X) <x, <x, <1... < 4, <x, = b, Ax; = x; -— x, and x) S X, S x; 
Geometrically, we can interpret this integral as the area between the graph y = f(x), the 
x axis and the lines x = a and x = b, as illustrated in Figure 3.13. 


YA 
y=f(x) 


~V¥ 


Line integrals 


Consider the integral 


| f(x, y)dx, where y = g(x) 


b 


This can be evaluated in the usual way by first substituting for y in terms of x in the 
integrand and then performing the integration 


| Sx, g(x) dx 


a 


Clearly the value of the integral will, in general, depend on the function y = g(x). It may 
be interpreted as evaluating the integral f ‘ F(x, y)dx along the curve y = g(x), as shown 
in Figure 3.14. Note, however, that the integral is not represented in this case by the 
area under the curve. This type of integral is called a line integral. 

There are many different types of such integrals, for example 


B B és B 
| f(x, y) dx, | f(x, y)ds, | f(x, y) dt, | [fi(x, y)dx+filx, y)dy] 
A A t A 
Cc G Cc Cc 

Here the letter under the integral sign indicates that the integral is evaluated along the 
curve (or path) C. This path is not restricted to two dimensions, and may be in as many 
dimensions as we please. It is normal to omit the points A and B, since they are usually 
implicit in the specification of C. 


www.2Ofile.org 


216 VECTOR CALCULUS 


Example 3.17 


Figure 3.15 
Portion of circle. 


Solution 


Example 3.18 


Solution 


Evaluate fxy dx from A(1, 0) to B(O, 1) along the curve C that is the portion of x* + y?= 1 
in the first quadrant. 


y 
B 
0,1 
(0, 1) B 
A 
re) (1,0) x 


The curve C is the first quadrant of the unit circle as shown in Figure 3.15. On the curve, 
y= \(1 — x’), so that 


0 

[vas = | xy(1-x’)dx =[-13d-»)"Y} =-! 
1 

(ol 


Evaluate the integral 


I= [ter r2nareerty an 


a 


from A(0, 1) to B(2, 3) along the curve C defined by y=x + 1. 


The curve C is the straight line y= x + 1 from the point A(0, 1) to the point B(2, 3). 
In this case we can eliminate either x or y. Using 


y=x+1 and dy=dx 


we have, on eliminating y, 


x=2 
i-| {[x° + 2(x+ 1)]dx+ [xt (x+1)’]dx} 


x=0 


2 
i | (2x" + 5x +3)dx = [3x° + $x" + 3x] = $ 
0 


In many practical problems line integrals involving vectors occur. Let P(r) be a point 
on a curve C in three dimensions, and let t be the unit tangent vector at P in the 
sense of the integration (that is, in the sense of increasing arclength s), as indicated in 
Figure 3.16. Then tds is the vector element of arc at P, and 


tds= dx, dy 54 ey ds=dxi+dyj+dzk=dr 
ds ds ds 
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Figure 3.16 
Element of arclength. 


Iffi(x, y, Z), AM, y, z) and f(x, y, z) are the scalar components of a vector field F(r) then 


| [hi(x, y, z)dx + f(x, y, z)dy +fa(x, y, 2) dz] 


=| |fony 2 Xds+fca,y, 2) Past for y, 2 ads 
. ds ds ds 


=| F-tds | F-dr 
Cc GC 


Thus, given a vector field F(r), we can evaluate line integrals of the form {. F «dr. In 
order to make it clear that we are integrating along a curve, the line integral is some- 
times written as [. F- ds, where ds = dr (some authors use d/ instead of ds in order to 
avoid confusion with dS, the element of surface area). In a similar manner we can 
evaluate line integrals of the form J. F x dr. 


Example 3.19 — Calculate (a) {..F- dr and (b) fF x dr, where C is the part of the spiral r= (acos 0, 
asin 0, a@) corresponding to0 = 8S $1, and F=r7i. 


Solution The curve C is illustrated in Figure 3.17. 
(a) Since r=acos 0i+ asin Oj + aOk, 
dr =—asin 0d0i + acos 0d0j + adOk 
so that 
F-dr=r'i-(-asin 0d0i + acos 0d0j + ad@k) 
=—ar’ sin 0d0 
=—a'(cos’*6 + sin’?@+ &) sin dO =—a(1 + 6) sin 6d0 


since r = |r| = \(a’ cos’6 + a’ sin’6 + a’). Thus, 


Figure 3.17 


The spiral n/2 
r= (acos 6, F-dr=-a | (1+@)sin0@d0 
asin 0, a@). a 


0 
= -a°[cos 0+2@sin 0 - 6 cos Alo. , using integration by parts 


= -a*(n — 1) 
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i j k 
(b) Fxdr= r 0 0 
-—asin@d@ acos@d@ adé@ 
= —ar’ dj + ar’ cos 0d0k 


-—a@(1+ @)d0j+aX(1 + &) cos OdOk 
so that 


n/2 m/2 
| rxar=-ie'| (14+ 600-+4e"| (1+ @)cos@d0 
Cc 


0 0 


3 3 
Ta 25 «, 2 2 
=—"2 (12 fn? - A)k 
ag 12th J+ Se - 4) 


F(r) The work done as the point of application of a force F moves along a given path 
t C as illustrated in Figure 3.18 can be expressed as a line integral. The work done 
; eee ; — . 
€ as the point of application moves from P(r) to P’(r + dr), where PP’= dr, is 
= dW =|dr||F|cos @= F- dr. Hence the total work done as P goes from A to B is 
Figure 3.18 Work done 
by a force F. v-| F- dr 
Cc 


In general, W depends on the path chosen. If, however, F(r) is such that F(r)- dr is an 
exact differential, say —dU, then W= J. - dU= U, — Ug, which depends only on A and 
B and is the same for all paths C joining A and B. Such a force is a conservative force, 
and U(r) is its potential energy, with F(r) =—grad U. Forces that do not have this prop- 
erty are said to be dissipative or non-conservative. 

Similarly, if v(r) represents the velocity field of a fluid then $, v-dr is the flow 
around the closed curve C in unit time. This is sometimes termed the net circulation 
integral of v. If $v -dr = 0 then the fluid is curl-free or irrotational, and in this case v 
has a potential function @(r) such that v = —grad @. 


3.4.2 Exercises 


56 Evaluate [yds along the parabola y = 2x from 59 IfA=(2y + 3)i+xzj + (yz—x)k, evaluate f- A-dr 
2 : : 
A(3, 23) to B(24, 4,6). [Recall: (*) ia (2) q along the following paths C: 
" ? (a) x=2P,y=14,z2=f fromt=0tor=1; 
57 Evaluate f® [2xy dx + (x? — y?) dy] along the arc (b) the straight lines from (0, 0, 0) to (0, 0, 1), 
of the circle x? + y* = 1 in the first quadrant from then to (0, 1, 1) and then to (2, 1, 1); 
A(1, 0) to B(0, 1). (c) the straight line joining (0, 0, 0) to (2, 1, 1). 
58 — Evaluate the integral [.V- dr, where 60 ‘Prove that F =(y*cosx + z*)i+ (2ysinx — 4)j 
V = (2yz + 3x°, vy’ + 4xz, 22” + 6xy), and C is the + (3xz? + z)k is a conservative force field. Hence 
curve with parametric equations x=, y=f,z=¢ find the work done in moving an object in this field 
joining the points (0, 0, 0) and (1, 1, 1). from (0, 1, —1) to (7/2, -1, 2). 
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61 Find the work done in moving a particle in the force any curve C joining the point (0, 0, 0) to the 
field F = 3x7i + (2xz — y)j + zk along point (1, 2, 3). 
2_ 3 
(a) the curve defined by x" = 4y, 3x" = 82 from 63 If F=xyi-zj+xk and Cis the curve x=", y = 21, 
x=0 Hoe = 2; z=? from t=0 to t= I, evaluate the vector line 
(b) the straight line from (0, 0, 0) to (2, 1, 3). integral [.F x dr. 
(c) Does this mean that F is a conservative force? 
Give reasons for your answer. 64 IfA=(3x+y, -x, y—z) and B = (2, -3, 1) 
evaluate the line integral $.(A x B) x dr around 
62 Prove that the vector field F = (3x* — y, 2yz’ — x, the circle in the (x, y) plane having centre at the 
2y*z) is conservative, but not solenoidal. Hence origin and radius 2, traversed in the positive 
evaluate the scalar line integral [..F - dr along direction. 
3.4.3 Double integrals 
In the introduction to Section 3.4 we defined the definite integral of a function f(x) of 
one variable by the limit 
b n 
| J (x)dx= lim BS f(%;) Ax; 
: all Ax, 30 (=! 
where d=X%) <x,<x,<...<x,=b, Ax, =x;— 4x, and x,_, SX, < x;. This integral 
is represented by the area between the curve y = f(x) and the x axis and between x = a 
and x = b, as shown in Figure 3.13. 
Now consider z = f(x, vy) and a region R of the (x, y) plane, as shown in Figure 3.19. 
Define the integral of f(x, y) over the region R by the limit 
|r y)dA= lim SS S(% is Fi) AA; 
neo i=l 
f all A4, 30 
where AA; (i= 1,..., 7) is a partition of R into n elements of area AA, and (¥,, ¥;) is 
a point in AA;. Now z = f(x, y) represents a surface, and so f(X;, ?,) AA; = Z;,AA, is the 
volume between z = 0 and z = Z, on the base AA,. The integral Jf, f(x, y) dA is the limit 
of the sum of all such volumes, and so it is the volume under the surface z = f(x, y) above 
the region R. 
Figure3.19 Volume Zz 


as an integral. 
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Ay; 


oO 


Figure 3.20 A possible grid for the partition of R Figure 3.21 Another possible grid for the partition of R 


(rectangular cartesian). (polar). 


The partition of R into elementary areas can be achieved using grid lines parallel to 
the x and y axes as shown in Figure 3.20. Then AA; = Ax, Ay,, and we can write 


[lr y)dA -|| J (x, y)dxdy = lim SG. Vi) Ax; Ay; 


R R 


Other partitions may be chosen, for example a polar grid as in Figure 3.21. Then the 
element of area is (7; A0,) Ar, = AA, and 


|[ y) dA =|) rece 0, rsin @)rdrdé@ (3.26) 


R R 


The expression for AA is more complicated when the grid lines do not intersect at right 
angles; we shall discuss this case in Section 3.4.5. 

We can evaluate integrals of the type Sf, f(x, y) dx dy as repeated single integrals in 
x and y. Consequently, they are usually called double integrals. 

Consider the region R shown in Figure 3.22, with boundary ACBD. Let the curve 
ACB be given by y = g,(x) and the curve ADB by y = g,(x). Then we can evaluate 
SS f(x, y) dx dy by summing for y first over the Ay,, holding x constant (x = X,, say), 
from y = g,(x,) to y = g,(x,), and then summing all such strips from A to B; that is, from 
x=atox=b. Thus we may write 


\| fyydd = lim S| SAS. y)Ay,| Ax, (n= min(n, n,)) 


all Ax;, Ay; 30 i=1 | j=l 


b V=8y(x) 
-| | fis nay} 
a LJ y=g,(x) 


Here the integral inside the brackets is evaluated first, integrating with respect to y, 
keeping the value of x fixed, and then the result of this integration is integrated with 
respect to x. 


R 
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Y = 82(x) 


R 


y= g(x) 
/ 


(a) 
Figure 3.22 The region R. 


Alternatively, we can sum for x first and then y. If the curve CAD is represented by 
x = h,(y) and the curve CBD by x = /,(y), we can write the integral as 


noo 
all Ay,, Ax,90 j=l | i=1 


d x=hy(y) 
“[[[rensle 
c x=h)(y) 
If the double integral exists then these two results are equal, and in going from one to 
the other we have changed the order of integration. Notice that the limits of integration 


are also changed in the process. Often, when evaluating an integral analytically, it is 
easier to perform the evaluation one way rather than the other. 


| | foy)d4= tim |S fe. 3) Ax] Ay (= min(n,, m)) 


R 


Example 3.20 —_ Evaluate Sf, (x? + y’) dA over the triangle with vertices at (0, 0), (2, 0) and (1, 1). 


Figure 3.23, Domain 
of integration for 
Example 3.20. 


Solution The domain of integration is shown in Figure 3.23(a). The triangle is bounded by the 
lines y= 0, y=x andy =2- x. 
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(a) Integrating with respect to x first, as indicated in Figure 3.23(b), gives 


[Jerre = | | (x +y°)dxdy 


R 


1 
= | [4x° +x] dy 


0 


il 
-| [$- 4y+4y’-Sy']dy=$ 


0 


(b) Integrating with respect to y first, as indicated in Figure 3.23(c), gives 


1 fy-* 2 (y=2-x 

\| (x +y") dA= | | of sy aydre | | (x +y") dydx 
OJ y=0 lJ y=0 

R 


Note that because the upper boundary of the region R has different equations for 
it along different parts, the integral has to be split up into convenient subintegrals. 
Evaluating the integrals we have 


1 fpy=x il 1 
| | (x? +y’) dydx = | [xy + i Toe -| tx? dx = t 


OJ y=0 0 0 


2 (y=2-x 2 
| | (x +y")dy dx = | [xy tty} uy dx 


1J y=0 1 


2 
2 3 
-| (2- 4x4 4x -$)dx =1 


1 


Thus 


| csraaeperes, as before 


R 


Clearly, in this example it is easier to integrate with respect to x first. 


Example 3.21 — Evaluate ff, (x + 2y) "dA over the region x — 2y < l andx > y’ +1. 


Figure 3.24 Domain 
of integration for 
Example 3.21. 
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Example 3.22 


Solution 
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The bounding curves intersect where 2y + 1 = y* + 1, which gives y = 0 (with x = 1) 
and y = 2 (with x = 5). The region R is shown in Figure 3.24. In this example we choose to 
take x first because the formula for the boundary is easier to deal with: x = y* + 1 rather 
than y = (x — 1)”. Thus we obtain 


2 p2y+l 
[fsa -| i (x+2y) 1? dx dy 
OJ y tl 


R 


2 
-| [2(x+ 2y))7 Jo 3" dy 


0 


2 
=| 2aysn!?-20+ 1a) 


0 


3/2 


= [}(4y+ 1)" - y’ - 2y], =? 


As indicated earlier, the evaluation of integrals over a domain R is not restricted 
to the use of rectangular cartesian coordinates (x, vy). Example 3.22 shows how polar 
coordinates can be used in some cases to simplify the analytical process. 


Evaluate ff,x°y dA, where R is the region x7 + y* < 1. 


The fact that the domain of integration is a circle suggests that polar coordinates are a 
natural choice for the integration process. Then, from (3.26), x =rcos 0, y=rsin @ and 
dé =rd@dr, and the integral becomes 


1 2n 
| | xyddA= | | r cos @ rsin@ rd@dr 
r=0/ 0=0 


R 
1 2n 
= | | r'cos°@ sin 80 d@dr 


r=0 4 @=0 


Note that in this example the integration is such that we can separate the variables r and 
6 and write 


1 20 
Ife dA = | | cos’@ sin @ d@ dr 
r=0 6=0 


R 


Furthermore, since the limits of integration with respect to @ do not involve r, we can 
write 


1 20 
Ife dA -| Har| cos’@ sin 0 d@ 
r=0 0=0 


R 
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65 


and the double integral in this case reduces to a product of integrals. Thus we obtain 


fey dA =[!r°], [-!cos’6]," = 0 
R 


Reflecting on the nature of the integrand and the domain of integration, this is the result 
one would anticipate. 


(z) There are several ways of evaluating double integrals using MATLAB. The simplest 
uses the command dblquad (f, x9, X;, Yo, y1)- For example, consider 


aS 
| | (x7 + y’)dx dy 
di O 


Here we define the integrand as an inline function 

i = ainkiime (22,92 & ye, "seh, Bee ys 
(Note that x is taken as a vector argument.) 

ie glole@wac (@ , 1, 2, © » 3) 
returns the answer 

kes 16 


For non-rectangular domains, the same command is used but the integrand is 
modified as shown below. Consider 


il fpee 
| | (x7 + y’) dx dy 
OJ 0 


from Example 3.20 (b). Here we define the integrand as the inline function 
* = dinlime (7 (Ge.82 £ wee) eyes <= ©)", RR", We 
where the logical expression (y - x <= 0) returns 1 if the expression is true and 
0 otherwise, so that the command 
i = clolewacl (e , @ , 1, © , dt) 
returns the required answer 
XE = @.23353 


despite integrating over a rectangular domain. 


3.4.4 Exercises 


Evaluate the following: 66 Evaluate 
3 2 3 5 
2 
(a) | | xy(xty)dydx — (b) | | xy dy dx |[fea 
oJ1 2/1 y 
1 2 
©| | (2x7 +y") dy dx over the rectangle bounded by the lines x = 0, 
= 23 x=2,y=l1andy=2. 
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68 


69 


70 


val 


72 


Evaluate ff (x? +’) dx dy over the region for which 
x20,y20andx+y<1. 


Sketch the domain of integration and evaluate 


(a) | ef (b) | ax] saiiay 


1 x % +y 


I \(=x") 1 
(c) | ax] renans Sa Fas 
0 J yew) (=x? -y) 


Evaluate ff sin Sax + y) dx dy over the triangle 
whose vertices are (0, 0), (2, 1), (1, 2). 


Sketch the domains of integration of the double 
integrals 


(a) [ «| —xvdy_ 


y 


(b) | a] (cos2y) \(1 - k sin’x) dx 


0 0 


Change the order of integration, and hence evaluate 
the integrals. 


Evaluate 


1 1 
| wy dx : 
0 vy viv. +x°)] 


Sketch the domain of integration of the double 
integral 


Lp y(-x’) 
[| —>— dydx 
oJ 0 VQ +y) 


3.4.5 


73 


74 


75 


76 


Green’s theorem in a plane 
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Express the integral in polar coordinates, and hence 
show that its value is +. 


Sketch the domain of integration of the double 
integral 


0 Ve +9") 


and evaluate the integral. 


Evaluate 


+ 
[ae s0e 
x+y +a 


over the portion of the first quadrant lying inside the 
circle x7 +? =a’. 


By using polar coordinates, evaluate the double 
integral 


2 2 
+ dx dy 
x+y 


over the region in the first quadrant bounded by the arc 
of the parabola y? = 4(1 — x) and the coordinate axes. 


By transforming to polar coordinates, show that the 
double integral 


[eeee 
(xy) 


taken over the area common to the two circles 
x?+y?=ax and x’? +y* = by is ab. 


This theorem shows the relationship between line integrals and double integrals, 
and will also provide a justification for the general change of variables in a double 


integral. 


Consider a simple closed curve, C, enclosing the region A as shown in Figure 3.25. If 
P(x, vy) and Q(x, y) are continuous functions with continuous partial derivatives then 


ferscro6n=|{(2- 


Cc A 
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Figure 3.25 Green’s 
theorem. 


Example 3.23 


Solution 


x= gi(y) 


x= g,(y) 


where C is traversed in the positive sense (that is, so that the bounded area is always on 
the left). This result is called Green’s theorem in a plane. 

The proof of this result is straightforward. Consider the first term on the right-hand 
side. Then, with reference to Figure 3.25, 


90 dP (g(x) 30 
[zee-f[ fee 
R oe 


| [O(g2(¥), ¥) - O(g1(), y)] dy 


= | oar oy 


LMN LKN 


= | O(x, y) dy = Os y)dy 


LMNKL c 
Similarly, 
-|| OP ax dy = p Pos y)dx 
dy 
A c 
and hence 
(2 : 4 dxdy = [P(x, y) dx + Q(x, y) dy] 
ox oy 


A 


An elementary application is shown in Example 3.23. 


Evaluate $ [2x(x + y) dx + (x* + xy + y’) dy] around the square with vertices at (0, 0), 
(1, 0), (1, 1) and (0, 1) illustrated in Figure 3.26. 


Here P(x, y) = 2x(x + y) and O(x, y) =x° + xy + y’, so that OP/dy = 2x, OO/ox = 2x + y 
and 0Q/ox — OP/oy = y. Thus the line integral transforms into an easy double integral 
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y 
(0,1) 1) pL2s(04y doe (8 tay?) = [Pree 
Cc 
(0, 0) | i ydxdy 
(1,0) * 
Figure 3.26 Path 
of integration for Pad a =! 
Example 3.23. : 


227 


It follows immediately from Green’s theorem (3.27) that the area A enclosed by the 


closed curve C is given by 


A= [free pow =-prdr= fey dred) 
- 


A c Cc 


Suppose that under a transformation of coordinates x = x(u, v) and y = y(u, v), the curve 


becomes C’, enclosing an area A’. Then 


y = = Ov avy 
A “72 ~ pu (2 ka dy) 
Cc 
ut) d( ov 
[a 2( ws) oy ( 2) ae 


Ou Ov. av Ou Ov av 
+u 
-|I{L Ee ay fe | Fae ox ayax Jj . 


A 


du dv du dv) 5 
=|(5 oy oy 5s] ee 


A 


This implies that the element of area du dv is equivalent to the element 


Ou dv _ Qu dv 
| & oy oy =) ad 


Here the modulus sign is introduced to preserve the orientation of the curve under the 


mapping. Similarly, we may prove that 


cea ae ade 


Hu, v) 


where 0(x, y)/O(u, v) is the Jacobian 


Ox Oy _ Ax By _ 
Qudv dvou Ax, y) 
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Example 3.24 


Solution 


Figure 3.27 

Domain of 
integration for 
Example 3.24: 

(a) in the (x, y) plane; 
(b) in the (u, v) plane. 


This enables us to make a general change of coordinates in a double integral: 


\| f(x, y) dxdy = |[ roa v), y(u, v))|J| dudv (3.29) 


where J’ is the region in the (uw, v) plane corresponding to A in the (x, y) plane. 
Note that the above discussion confirms the result 


Au, Vv) _ | A(x, y) . 
Ax,y) | Au, v) 


as shown in Section 3.1.3. Using (3.29), the result (3.26) when using polar coordinates 
is readily confirmed. 


Evaluate ff xy dx dy over the region in x = 0, y = 0 bounded by y =x? + 4, y=x’, 
y=6-x’ andy=12-x’. 


The domain of integration is shown in Figure 3.27(a). The bounding curves can be 
rewritten as y— x? = 4, y—x?=0, y+ x* = 6 and y + x’ = 12, so that a natural change 
of coordinates is to set 


u=ytx’, v=y-x° 


Under this transformation, the region of integration becomes the rectangle 6 < u < 12, 
0 <v <4, as shown in Figure 3.27(b). Thus since 


_ Ax, y) _ | Au, v) ae 4 
Se) Au, v) Ee 4 ~ Ax 


the integral simplifies to 


|| xy dx dy = || xy dud 


A A’ 


YA 

12 

6 vi 
4-+- 

_ «UF 
Oo 6 12 * 


(a) (b) 
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Three-dimensional 
generalization of 
Green’s theorem. 


77 


3.4 TOPICS IN INTEGRATION 229 


Hence 


|| xy dxdy =} |] rauao = | (u+v)dudv, since y=(u+v)/2 


A A’ 
4 12 
=i co (ut+v)du = 33 
0 6 


We remark in passing that Green’s theorem in a plane may be generalized to three 
dimensions. Note that the result (3.27) may be written as 


is 


per O,0)-dr= || curl[(P, O, 0)]-kdxdy 


G A 


For a general surface S with bounding curve C as shown in Figure 3.28 this identity 
becomes 


p F(r): dr= \| curl F(r)- dS 


(e Ss 


where d.S = fidS is the vector element of surface area and fi is a unit vector along the 
normal. This generalization is called Stokes’ theorem, and will be discussed in 
Section 3.4.12 after we have formally introduced the concept of a surface integral. 


Surface S 


3.4.6 Exercises 


Evaluate the line integral Verify your answer using Green’s theorem in a plane. 


p sin dre e08 


C 


taken in the anticlockwise sense, where C is the 


78 Use Green’s theorem in a plane to evaluate 


f [(xy? - y) dx + (x+y") dy] 


Cc 


perimeter of the triangle formed by the lines as a double integral, where C is the triangle with 


1 
Vaz, 


vertices at (0, 0), (2, 0) and (2, 2) and is traversed 


= _ ; : bee eae 
Yrzt, x=0 in the anticlockwise direction. 
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79 ~~ Evaluate the line integral 81 Evaluate 


a 2a-x 

I= o (xy dx+x dy) | ex] a dy 
0 y Aa t(ytxy 

a 

using the transformation of coordinates u = x + y, 


where C is the closed curve consisting of y =x? v=x-y 


from x = 0 tox = 1 and y= \x from x = 1 tox =0. 
Confirm your answer by applying Green’s theorem 82 Using the transformation 
in the plane and evaluating / as a double integral. 


= y 
+y=u, 5 
80 Use Green’s theorem in a plane to evaluate the line i ae ad 
integral 
integra show that 
[(e'- 3y°) dx +(e” + 4x°) dy] ee oi oe is 
- x , y x+y u 
i‘ ? | ay Ete ar= | au| « dv=e-1 
C 0 y x 0 0 


where C is the circle x? + y* = 4. (Hint: use polar 
coordinates to evaluate the double integral.) 


3.4.7. Surface integrals 


The extensions of the idea of an integral to line and double integrals are not the only 
generalizations that can be made. We can also extend the idea to integration over a 
general surface S. Two types of such integrals occur: 


(a) |[ re, z) dS 


Ss 


(b) \| ro -iss=|| F(r)- dS 


Ss Ss 


In case (a) we have a scalar field f(r) and in case (b) a vector field F(r). Note that 
dS = fAidS is the vector element of area, where fi is the unit outward-drawn normal 
vector to the element dS. 
In general, the surface S can be described in terms of two parameters, u and v say, 
A so that on S 
C,(r(u, Vo)) 


r=r(u, v) = (x(u, v), v(u, v), Z(u, V)) 


The surface S can be specified by a scalar point function C(r) = c, where c is a 
constant. Curves may be drawn on that surface, and in particular if we fix the value of 
C(r(u,v)) One of the two parameters wu and v then we obtain two families of curves. On one, 


CAr (Mo, v)) C(r(u, vo)), the value of u varies while v is fixed, and on the other, C,(r(u, v)), the 
Figure 3.29 Parametric Value of v varies while wu is fixed, as shown in Figure 3.29. Then as indicated in 
curves on a surface. Figure 3.29, the vector element of area dS is given by 
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dg OF pCa, On ce 


Ou Ov a Ov 
-(# dy 2) «(2 a dv SE) dudv = (hit Jnj+Jsk) dudv 


du dv 


ou ou’ Ou ov ov da 
where 
dv dz daz yp dedy dex, _dxdv Ord Ga 
‘ Quav. avoau’ > dudv dvau’ > Oudv dvau ° 
Hence 
|r ; as= || (PJ, + QJ + RJy) du dv 
AY A 
| | FG, y, ZAS = | | f(u, v) (i + J3 + J3) du dv 
AY A 


where F(r) = (P, Q, R) and A is the region of the (uw, v) plane corresponding to S. Here, 
of course, the terms in the integrands have to be expressed in terms of u and v. 

In particular, w and v can be chosen as any two of x, y and z. For example, if z = z(x, y) 
describes a surface as in Figure 3.30 then 


r= (x, y, 2(x, y)) 


with x and y as independent variables. This gives 


er mene J,=1 
and 
| F(r)- dS = | (r% -0& a 5 +R) dxdy (3.31a) 
A 
= az\ azyY 
HCE Aas = HG 2 Ce I) 1+) = dxdy (3.31b) 
ox ny 
5 A 
Figure 3.30 A surface Zz Surface S 
described by Element dS 


z=2z(x, y). 


<Y 
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Example 3.25 


Figure 3.31 

(a) Surface S for 
Example 3.25; 
(b) quadrant of a 
circle in the (x, y) 
plane. 


Solution 


Evaluate the surface integral 


i (x+y+z)dS 


Ss 


where S is the portion of the sphere x” + y* + z” = 1 that lies in the first quadrant. 


The surface S is illustrated in Figure 3.31(a). Taking 


z= (1-x’—y’) 


we have 
Oz _ KR OF 
dx \(l--y) ay (=x -y’) 
giving 
1+(2) +( 2) _ x+y t+(1-x'-y’) 
ox oy (1-x’-y’) 
7 1 
Vl -x"-y’) 


Using (3.17) then gives 


|[orrenas= [[ tree ca-e vy ares 
y-x -y) 


Ss A 


where A is the quadrant of a circle in the (x, y) plane illustrated in Figure 3.31(b). 
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Thus 


yd =2°) 

|| cryenas - ax] Peer x +—_+—~ + l|dy 
lex yy) sax ay) 

S 


(l=) 
‘| xsin ‘(5)- (=x -y’) +y dx 


(tex) 


0 


lp +2\(1-x sya 


1 


= np -x°)+ sins 
4 0 


8, 
= jn 


An alternative approach to evaluating the surface integral in Example 3.25 is to evaluate 
it directly over the surface of the sphere using spherical polar coordinates. As illustrated 


in Figure 3.32, on the surface of a sphere of radius a we have 
x = asin @ cos @, y=asin@ sino 
Z=lacosig) dS =a’ sin@ dé d@ 
Figure 3.32 Surface 


element in spherical 
polar coordinates. 


asin@ 


In the sphere of Example 3.25 the radius a = 1, so that 


n/2 pn/2 
|[cerenas= | | (sin 8 cos @+ sin 8 sin@ + cos @) sin 0 dO. d@ 


0 0 
S 


n/2 
-| [i;ncoso+insin@+}]do =? 


0 


as determined in Example 3.25. 
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In a similar manner, when evaluating surface integrals over the surface of a cylinder of 
radius a, we have, as illustrated in Figure 3.33, 


x =acos @, y=asin@, B=, dS=adzdo 


Figure 3.33 
Surface element 
in cylindrical 
polar coordinates. 


Example 3.26 __ Find the surface area of the torus shown in Figure 3.34(a) formed by rotating a circle 
of radius b about an axis distance a from its centre. 


Figure 3.34 (a) torus Zz 
of Example 3.26; i Locus of 
(b) position vector of a _ = oa Z centre of circle 


point on the surface of 
the torus. 


/ 


Nae 
\/ (a + bcos d) cos 8 


(a) 


Solution From Figure 3.34(b), the position vector r of a point on the surface is given by 
r=(a+ bcos 6)cos 6i+ (a+ bcos ¢) sin Oj + bsin ok 


(Notice that 0 and @ are not the angles used for spherical polar coordinates.) Thus 
using 3.16, 


J, = (a+ bcos @) cos 0(b cos @) — (—b sin @ sin 6)(0) 
J, = (0)(—b sin @ cos 8) — (bcos 6)(a + bcos ¢)(—sin 8) 
J; =—-(a + bcos @) sin @(—b sin @ sin 8) — (—b sin @ cos 8)(a + bcos ¢) cos 8 
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Simplifying, we obtain 
J, = b(a + bcos @) cos 0 cos @ 
J, = b(a + bcos ¢) sin 6 cos 
J, = b(a + bcos $) sind 


and the surface area is given by 


2n (20 
s=|'| \(J + J3 + J3) dO do 


-| | b(a + bcos @) d0d@ 


0 0 
= 4 ab 


Thus the surface area of the torus is the product of the circumferences of the two circles 
that generate it. 


Evaluate ff;V- dS, where V = zi + xj — 3y*zk and S is the surface of the cylinder 
x’ + y’ = 16 in the first octant between z = 0 and z= S. 


The surface S is illustrated in Figure 3.35. From Section 3.2.1, the outward normal to 
the surface is in the direction of the vector 


n= grad (x* + y* — 16) = 2xi + 2yj 
so that the unit outward normal fi is given by 
oe 2xi+ 2yj 
2x +) 
Hence on the surface x* + y* = 16, 
A= i Git yf) 
giving 
dS = dSf = }dS(xi+y/) 


Projecting the element of surface dS onto the (x, z) plane as illustrated in Figure 3.35, 
the area dx dz of the projected element is given by 


dxdz=dScosB 


where f is the angle between the normal fi to the surface element and the normal j to 
the (x, z) plane. Thus 


dxdz=dS|A-j|=4dS|Git+ sj) -jl=idSy 
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Figure 3.35 
Surface S for 
Example 3.26. 


x 
(4, 0, 0) 


giving 
dS =4dx dz 
y 
Also, 
V-dS=V-AdS=(Gi+xf- 3y'zh): (EMA ards = R24) ay dz 
y y 


so that 


|[r-as=[[ =e aa: 
y 


Ss A 


where 4 is the rectangular region in the (x, z) plane bounded by0 Sx $4,052 <5. 
Noting that the integrand is still evaluated on the surface, we can write y = \(16 — x’), 
so that 


|[v-os= [| b+ see 
oJ0 \(16 - x") 


Ss 


4 2 5 
=| xz-+—+— dx 
of 2y(16 =") 


0 


4 
=| 5x+—2% _ dx 
of 2V06-2*) 


= [$x°- 816-2’) ], 


= 90 


An alternative approach in this case is to evaluate ff, (xz + xy) dS directly over 
the surface using cylindrical polar coordinates. This is left as Exercise 90, in Exer- 
cises 3.4.8. 
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Evaluate the area of the surface z = 2 — x* — y* lying 87 Evaluate the surface integral ff, U(x, y, z) dS, 
above the (x, vy) plane. (Hint: Use polar coordinates where S is the surface of the paraboloid 
to evaluate the double integral.) z=2-(x? +’) above the (x, y) plane and 
U(x, y, Z) is given by 
E 1 t eA 
ee (a) 1 (b) x24? (Cz 
(a) Jfs(x? +7) dS, where S is the surface area of ; ee Seats 
the plane 2x + y+ 22 = 6 cut off by the planes Give a physical interpretation in each case. 
z=0,2=2,y=0,y=3; : 
(i (id 5 whee as dhe sudeemererihe 88 Determine the surface area of the plane 
: 2.42422 2x +y+2z = 16 cut off by x =0, v=0 
hemisphere x° + y° + z° = | (z > 0) cut off dPagt see 
by the cylinder x* -x + y?=0. = 
89 Show that the area of that portion of the surface 
Evaluat : her 
vatuale heed Sines of the paraboloid x? + y? = 4z included between 
(a) v=(xy, -x*, x +z) and S is the part of the planes z = | and z= 3 is = m4 — 2). 
the plane 2x + 2y + z = 6 included in the 
first octant; 90 Evaluate the surface integral in Example 3.27 using 
(b) v = (3y, 2x’, z*) and S is the surface of the cylindrical polar coordinates. 
cylinder x°+y°=1,0<z<1, 
91 If F=yi + (x — 2xz)j — xyk, evaluate the surface 


Show that ff,z? dS = 2T, where S is the surface of 
the sphere x7 +? +z°=1,z20. 


3.4.9 Volume integrals 


integral ff,(curl F)- dS, where S is the surface of 
the sphere x7 +? +z*=a’,z=0. 


In Section 3.4.7 we defined the integral of a function over a curved surface in three 
dimensions. This idea can be extended to define the integral of a function of three 
variables through a region 7 of three-dimensional space by the limit 


||[ronoer- lim ¥ fF. 200, 


ip 


all AV; 30 


i=1 


where AV, (i= 1,..., 7) is a partition of T into n elements of volume, and (X,, ¥,, Z;) is 
a point in AV, as illustrated in Figure 3.36. 
In terms of rectangular cartesian coordinates the triple integral can, as illustrated in 


Figure 3.37, be written as 


[[] ron2er= [a] 


T 


£(x) 


g(x) 


hy(x, y) 
ev F(x, y, Z) dz 


(3.32) 


h 1 (x,y) 


Note that there are six different orders in which the integration in (3.32) can be 


carried out. 


As we saw for double integrals in (3.28), the expression for the element of volume 


dV = dx dy dz under the transformation x = x(u, v, w), y= y(u, Vv, W), Z=Z(u, v, Ww) may 
be obtained using the Jacobian 
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Upper surface z = h,(x, y) 


a | Y= 82x) 
y=g@)  — f 
Projection of 
volume onto 
(x, y) plane 
Figure 3.36 Partition of region T into Figure 3.37 The volume integral in terms of rectangular 
volume elements AV;,. cartesian coordinates. 
ox Ww & 
du ou du 
yu 2@yz)_|dx Ww & 
O(u, v, w) CUO DMOl) 
ox gi & 
dw dw daw 
as 
dV = dxdydz=|J|dududw (3.33) 
For example, in the case of cylindrical polar coordinates 
x = pcos ¢, y=psin @, Z=Z 
cos@ sind 0 
J=p|-sind cos@ O}=p— 
0 0 1 
so that 
dV = pdp dg dz (3.34) 
a result illustrated in Figure 3.38. 
Similarly, for spherical polar coordinates (7, 6, @) 
x=rsin cos @, y=rsin@sin@, z=rcos0 
sinO@ cos @ sin @ sing cos 0 
J= |rcos@cos@ rcos@sing -rsind| = r sin@ 
-rsin@singd rsin@cos¢d 0 
so that 
dV =r’ sin @drd@d@ (3.35) 


a result illustrated in Figure 3.39. 
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Figure 3.38 Volume element in Figure 3.39 Volume element in spherical 
cylindrical polar coordinates. polar coordinates. 


Example 3.28 Find the volume and the coordinates of the centroid of the tetrahedron defined by x = 0, 
y20,z20andx+yt+z<1. 


Solution The tetrahedron is shown in Figure 3.40. Its volume is 


x=1 y=1l-x z=l1-x-y 
ee ae V= |] dvdyde =| to w| dz 
2 x=0 y=0 z=0 


tetrahedron. 


1 ie 1 
-| «| (-x-ydy= | fa tare! 
0 0 0 


a. +Y=1, Tet the coordinates of the centroid be (x, ¥, Z); then, taking moments about the line x = 0, 
Z=2, 
Figure 3.40 
Tetrahedron for 
Example 3.28. xV= xdV = x dx dy dz 
tetrahedron tetrahedron 
1 1-x 1-x-y 1 
-| «| w| vde=| bx de = 4 
0 0 0 0 
Hence ¥ = t, and by symmetry y =Z = ie 


Example 3.29 __ Find the moment of inertia of a uniform sphere of mass M and radius a about a diameter. 


Solution A sphere of radius a has volume 41a°/3, so that its density is 3M/4na’. Then the moment 
of inertia of the sphere about the z axis is 


hs 3M (x + y’) dx dy dz 
4ta 


sphere 
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In this example it is natural to use spherical polar coordinates, so that 


1 = 3 ||| (7 sin’9) sind dr d0 do 
4na 


sphere 


a Tw 20 
= 31 ar} sin'ode| do = 24 (10')(4)(20) 
4nta J, F 7 4na 


2 
= 2Ma 


(2) Evaluating triple integrals using MATLAB uses the command triplequad. For 
example, consider (see Example 3.28): 


iL fpPtlese fp ikereaay 
| | | x dx dy dz 
0/0 0 


Here we write the integrand as the inline function 

iy ce samba (se, ( Se ay = os eal) ?, Ue, Mer, Ge" he 
so that the command 

t= twreiolequec ( , © , Lt, O , I, ©, i) 
returns the answer 

zk 2 0 .@416 


This procedure could be slow because of the large number of points at which the 
integrand is evaluated. 


3.4.10 Exercises 


92 Evaluate the triple integrals 95 Evaluate Sff, xyzdxdydz, where V is the region 
bounded by the planes x = 0, y= 0, z= 0 and 


: : xt+yt+z=l1 
«| | «| xyzdz . ; 
0 0 1 96 


Sketch the region contained between the parabolic 
cylinders y = x* and x = y’ and the planes z = 0 and 


2 73 £4 
(b) | | | xyz’ dz dy dx x+y+z=2. Show that the volume of the region 


ol xls may be expressed as the triple integral 


1 fyx f2-x-y 
93 Show that | | dz dy dx 
osx J0 


1 Zz REZ if 
t. 
| | ex| eee er, and evaluate i 
= o noe 97 Use spherical polar coordinates to evaluate 
94 Evaluate Sff sin (x + y +z) dx dy dz over the x(x +? + 2) dxdydz 
portion of the positive octant cut off by the plane 
xty+z=T. v 
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where V is the region in the first octant lying within 
the sphere x7 ++? +27=1. 
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where V is the volume of the tetrahedron 
bounded by the planes x = 0, y = 0, z = 0 and 
x+yt+z=1. 


98 Evaluate fff x°y2z*(x + y +z) dx dy dz throughout 
the region defined byx+y+z<1,x>0,y>0, 190 Evaluate SSS -yzdx dy dz taken throughout the 
a", prism with sides parallel to the z axis, whose base 
is the triangle with vertices at (0, 0, 0), (1, 0, 0), 
99 Show that ifx+y+z=u,y+z= wand z= uw (0, 1, 0) and whose top is the triangle with vertices 
then at (0, 0, 2), (1, 0, 1), (0, 1, 1). Find also the position 
of the centroid of this prism. 
OXY, 2) _ on 
O(u, v, W) 
101 Evaluate fff z dx dy dz throughout the region 
Hence evaluate the triple integral defined by x? +y?<27,x°4+y° +2? <1,2>0. 
102 Using spherical polar coordinates, evaluate 


{ll exp[-(x + y + z)’]dxdydz 


V 


JSSf x dx dy dz throughout the positive octant of 
the sphere x7 +? + 2° =a’. 


3.4.11 Gauss’s divergence theorem 


In the same way that Green’s theorem relates surface and line integrals, Gauss’s theorem 
relates surface and volume integrals. 

Consider the closed volume V with surface area S shown in Figure 3.41. The surface 
integral ff, F -dS may be interpreted as the flow of a liquid with velocity field F(r) 
out of the volume V. In Section 3.3.1 we saw that the divergence of F could be 
expressed as 


Figure 3.41 
Closed volume V 
with surface S. 


div F=V-F = lim Howoutof AV 
AV>0 AV 


In terms of differentials, this may be written 
div F dV = flow out of dV 


Consider now a partition of the volume V given by AV; (i= 1,..., 7). Then the total 
flow out of V is the sum of the flows out of each AV;. That is, 


|| Fs lim s (flow out of AV;) = lim y (div FAV;) 
neo i=1 n—-eoo i=1 


Ss 


giving 


(ee 


Ss ie 


(3.36) 


This result is known as the divergence theorem or Gauss’s theorem. It enables us 
to convert surface integrals into volume integrals, and often simplifies their evaluation. 
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Example 3.30 _—_A vector field F(r) is given by 


F(r) =xtyit x°y?j + x°yzk 


Find ff; F-dS, where S is the surface of the region in the first octant for which 
xX+y4+z<l. 


Figure 3.42 Region V 
and surface S for 
Example 3.30. 


Solution We begin by sketching the region V enclosed by S, as shown in Figure 3.42. It is clear that 
evaluating the surface integral directly will be rather clumsy, involving four separate 
integrals (one over each of the four surfaces). It is simpler in this case to transform it into 
a volume integral using the divergence theorem (3.36): 


[[rs- ||] swear 


7 
Here 
div F = 3x7y + 2x*y + x°y = 6x7y 


and we obtain 


1 1-x 1-x-y 
|[F-os= | | wv 6x'y dz 
0 0 0 
Ss 


1 1-x l=-x-y 
=6 | x dx | ydy | dz 
0 0 0 (see Example 3.28) 


if l-x 
= | vas [(1 — x)y — y'] dy 


1 
-| x (1 —x)dx = + 


0 


Example 3.31 Verify the divergence theorem 


[[e-ac= | avvar 


V 


when F = 2xzi + yzj + z’k and V is the volume enclosed by the upper hemisphere 
etyt+r=ai,z20. 
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Figure 3.43 
Hemisphere for 
Example 3.31. 


Solution 
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The volume V and surface S of the hemisphere are illustrated in Figure 3.43. Note that 
since the theorem relates to a closed volume, the surface S consists of the flat circular 
base in the (x, y) plane as well as the hemispherical surface. In this case 


div F =2z+2z+2z=5z 


so that the volume integral is readily evaluated as 


{{] 5zdxdydz = | Szmr dz = | 5mz(a° — z’)dz = sna" 
0 0 


ig 


Considering the surface integral 


[ease |[ emacs [] teas 


Ss circular base hemisphere 
The unit normal to the base is clearly 7, =—k, so 
F-f,=-z° 


giving 


|| Faas=o 


circular base 


since z = 0 on this surface. 
The hemispherical surface is given by 


fay 2D=r+yt+z7-a=0 
so the outward unit normal i, is 


et Vf_ _ 2xi + 2yj + 2zk 
— = 
|Vf| 2° +y + Z’) 


Since x? + y? +z? =a’ on the surface, 


A,=~i+rt jr ik 
a a a 
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giving 


2 2 3 2 
Ff, = 224 U% 42 = Mp2 te yt) 
a a a a 


\| F-f,dS = \| 2 (x? + a’) dS 
a 


hemisphere hemisphere 


Hence 


since x° + y* +.z”= a’ on the surface. Transforming to spherical polar coordinates, 
x=asinOcos¢, z=acos0, dS=a’ sinddédg 
the surface integral becomes 
2n fn/2 
F-f,dS = “| | (sin @ cos @ + sin’@ cos @ cos’) dé do 


0 0 
hemisphere 


20 
: 4 
“| [}sin’@ + jsin’@ cos o]s? do 


0 


20 
a “| [5 +4 cos @]d@ = ina’ 


0 


thus confirming that 


| r-as=|[] awear 


Ss Vv 


3.4.12 Stokes’ theorem 


Stokes’ theorem is the generalization of Green’s theorem, and relates line integrals in 
three dimensions with surface integrals. At the end of Section 3.3.3 we saw that the curl 


of the vector F could be expressed in the form 


~_ 4:.. flow round AS 
ia a a 


In terms of differentials, this becomes 


curl F'- dS = flow round dS 


can write 


pr dr = lim >, (flow round AS;) = lim b3 (curl F- AS;) 
i=l pe i=1 


no & 


Figure 3.44 Surface S 
bounded by curve C. Cc 
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Consider the surface S shown in Figure 3.44, bounded by the curve C. Then the 
line integral ¢, F-dr can be interpreted as the total flow of a fluid with velocity field 
F around the curve C. Partitioning the surface S into elements AS; (j= 1,..., 7), we 


Cc 
Figure 3.45 Two 
paths, C; and C), 
joining points A 
and B. 


so that 


p pdr || (cul F)- a8 


Ss 
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(3.37) 


This result is known as Stokes’ theorem. It provides a condition for a line integral to 
be independent of its path of integration. For, if the integral [2 F-dr is independent of 


the path of integration then 


[rare [Por 


Cc) Cy 


where C;, and C, are two different paths joining A and B as shown in Figure 3.45. Since 


[rare | rw 


C, =; 


where —C, is the path C, traversed in the opposite direction, we have 


[rans | rare 


Cc, 7c, 


That is, 


prar=0 


iC 


where C is the combined, closed curve formed 
implies that if 6. F- dr = 0 then 


| (curl F) - dS = 0 


Ss 


from C, and —C,. Stokes’ theorem 


for any surface S' bounded by C. Since this is true for all surfaces bounded by C, we 
deduce that the integrand must be zero, that is curl F = 0. Writing F = (F), F5, F), we 


then have that 
F-dr=F,dx+F,dy+F, dz 

is an exact differential if curl F = 0; that is, if 
OF, _ dF, F,_ OF, Fy_ F, 
Oz Ox’ Oy Ox’ oz oy 

Thus there is a function f(x, y, z) = f(r) such that 


- of med - 
F, Ox F, oy FP, Oz 


that is, such that F(r) = grad f- 
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When F(r) represents a field of force, the field is said to be conservative (since it 
conserves rather than dissipates energy). When F(r) represents a velocity field for a 
fluid, the field is said to be curl-free or irrotational. 


Example 3.32 Verify Stokes’ theorem for F = (2x — y)i — yz’j — yzk, where S is the upper half of the 
sphere x* + y+ z* = 1 and C is its boundary. 


Figure 3.46 
Hemispherical 
surface and boundary 
for Example 3.32. 


x 


Solution The surface and boundary involved are illustrated in Figure 3.46. We are required to 


show that 
pear [| cu Fas 
iG Ss 


Since C is a circle of unit radius in the (x, y) plane, to evaluate 6. F- dr, we take 
x= cos @, y=sing 
so that 
r=cos gi+ sin gj 
giving 
dr =—sin @ ddi + cos 6 doj 
Also, on the boundary C, z = 0, so that 
F=(2x—y)i=(2cos @- sin @)i 
Thus 


prear=| (2 cos @ — sin @)i- (-sin gi + cos dj) dd 


2n 


-| (-2 sin @ cos @ + sin’o) do -| [-sin 26 + $(1 + cos 2@)] do 


0 0 


=T 
i J k 
Q Q Q 
curl F = A a9 ae =k 
2 2 
2x-y -yZ -yz 
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The unit outward-drawn normal at a point (x, y, z) on the hemisphere is given by 
(xi + yj + zk), since x* + y* +z? = 1. Thus 


|[ out was || k-(xi + yj + zk) dS 


Ss Ss 


[° fe. 
0 0 


= 2n[} sin’@ | 


0 


m/2 


cos@ sin@ ddd@ 


Hence $, F-dr = ff;(curl F)-dS, and Stokes’ theorem is verified. 


3.4.13 Exercises 


Evaluate ff, F-dS, where F = (4xz, —y’, yz) and Sis 
the surface of the cube bounded by the planes x = 0, 
x=l1,y=0,y=1,z=0Oandz=1. 


Use the divergence theorem to evaluate the surface 
integral [fF - dS, where F =xzi + yzj +z°k and S is 
the closed surface of the hemisphere x* + y?+z?=4, 
z > 0. (Note that you are not required to verify the 
theorem.) 


Verify the divergence theorem 


[ro [fore 


S Vv 
for F = 4xi — 2y?j + z’k over the region bounded by 
x? t+y?=4,7=0andz=3. 


Prove that 


{{] (grad @) - (curl F) dV = \| (F x grad @)- dS 


ie Ss 


Verify the divergence theorem for F = (xy +y°)i+x°yjf 
and the volume V in the first octant bounded by 
x=0,y=0,2=0,z=1andx?+y’=4. 


Use Stokes’ theorem to show that the value of the 
line integral [% F-dr for 
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110 


Wala 


Wile 


F = (36xz + 6y cosx, 3+ 6sinx+zsiny, 
18x? — cos y) 


is independent of the path joining the points A and B. 


Use Stokes’ theorem to evaluate the line integral 
f- A-dr, where A = —yi + xj and C is the boundary 
of the ellipse x7/a* + y7/b’ = 1, z=0. 


Verify Stokes’ theorem by evaluating both sides of 


[foun as = pra 


Ss Cc 


where F = (2x — y)i—z?j —y°zk and Sis the curved 
surface of the hemisphere x* + y? +z? = 16,2 = 0. 


By applying Stokes’ theorem to the function af(r), 
where a is a constant, deduce that 


\| (n x grad f) dS = | sar 


Ss GC 


Verify this result for the function f(r) = 3xy" and 
the rectangle in the plane z = 0 bounded by the 
lines x =0,x=1, y=0 and y=2. 


Verify Stokes’ theorem for F = (2y +z, x —z, y—x) 
for the part of x* + y? +z? = | lying in the positive 
octant. 
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See terdlistslalercar|yeierliveaa Streamlines in fluid dynamics 


As we mentioned in Section 3.1.5, differentials often occur in mathematical modelling 

of practical problems. An example occurs in fluid dynamics. Consider the case of 

steady-state incompressible fluid flow in two dimensions. Using rectangular cartesian 

coordinates (x, v) to describe a point in the fluid, let u and v be the velocities of the fluid 

in the x and y directions respectively. Then by considering the flow in and flow out of 

(x+Ax,y+Ay) a small rectangle, as shown in Figure 3.47, per unit time, we obtain a differential 
relationship between u(x, y) and v(x, y) that models the fact that no fluid is lost or gained 

Aye lS in the rectangle; that is, the fluid is conserved. 

The velocity of the fluid g is a vector point function. The values of its components 

u and v depend on the spatial coordinates x and y. The flow into the small rectangle in 


@, ¥) ae unit time is 
k— >! 
Ax u(x, V)Ay + v(&X, y)Ax 
Figure 3.47 where x lies between x and x + Ax, and 7 lies between y and y + Ay. Similarly, the flow 
Fluid flow. 


out of the rectangle is 
u(x + Ax, y)Ay + v(%, y + Ay)Ax 


where x lies between x and x + Ax and ¥ lies between y and y + Ay. Because no fluid is 
created or destroyed within the rectangle, we may equate these two expressions, giving 


u(x, » )Ay + v(X, y)Ax = u(x + Ax, p)Ay + 0%, y + Ay)Ax 


Rearranging, we have 


u(x + Ax, P) — u(x, y) 4 v(x, y + Ay) — v(x, y) = 


Ax Ay 
Letting Ax > 0 and Ay > 0 gives the continuity equation 
Streamline 
du, W_ 
ox oy 
> | Ay . ‘ 
The fluid actually flows along paths called streamlines so that there is no flow across 
x a streamline. Thus from Figure 3.48 we deduce that 
x 
vAx=uAy 
Figure 3.48 
Streamline. and hence 
vdx —udy=0 


The condition for this expression to be an exact differential is 
9 wy) -2(., 
ay”? = Ox u) 

or 


Ou, W_ 
ox oy 7 


This is satisfied for incompressible flow since it is just the continuity equation, so that 
we deduce that there is a function y(x, y), called the stream function, such that 
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Solution 
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pew and pena 
Ox 


It follows that if we are given wu and v, as functions of x and y, that satisfy the continuity 
equation then we can find the equations of the streamlines given by w(x, v) = constant. 


Find the stream function y(x, y) for the incompressible flow that is such that the velocity 
q at the point (x, y) is 


(yi(x? + y’), x/(x? + y)) 


From the definition of the stream function, we have 


ae 


u(x, y) = ay and Me es 


Ox 


provided that 


du, w 


‘Ox ae aa 


Here we have 


x ear 2 2 


so that 


2yx 


Ou _ __2xy d dv _ ats 
(x+y) 


ani 
ax (x +yy oy 


confirming that 


ou, Ov =f 
ox ‘Oy 
Integrating 
Q 
ot = -u(x,9) = = 
oy ge eine 


with respect to y, keeping x constant, gives 
w(x, y) = 5 In(@a’ + y’) + g@) 
Differentiating partially with respect to x gives 
O d 
OY —— 408 
ox x+y dx 
Since it is known that 


= wu y=5 


r+y 
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Figure 3.49 
Streamline illustrating 
a vortex. 


we have 


dg _ 4 
dx 


which on integrating gives 
BX)=C 

where C is a constant. Substituting back into the expression obtained for y(x, vy), we have 
wx, y) = ;In@’+y’)+C 


A streamline of the flow is given by the equation w(x, vy) = k, where k is a constant. 
After a little manipulation this gives 


xty=a and Ina=k-C 


and the corresponding streamlines are shown in Figure 3.49. This is an example of a 
vortex. 


vA 


S). 
7 


3.6 Engineering application: fithiatensice 


In modelling heat transfer problems we make use of three experimental laws. 


(1) Heat flows from hot regions to cold regions of a body. 

(2) The rate at which heat flows through a plane section drawn in a body is proportional 
to its area and to the temperature gradient normal to the section. 

(3) The quantity of heat in a body is proportional to its mass and to its temperature. 

In the simplest case we consider heat transfer in a medium for which the constants of 

proportionality in the above laws are independent of direction. Such a medium is called 


thermally isentropic. For any arbitrary region within such a medium we can obtain an 
equation that models such heat flows. The total amount Q(4) of heat within the region V is 


O(t) = || cpu(r, t) dV 


ie 
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where c is the specific heat of the medium, p is the density and u(r, #) is the temperature 
at the point r at time ¢. Heat flows out of the region through its bounding surface S. The 
experimental laws (1) and (2) above imply that the rate at which heat flows across an 
element AS of that surface is —-kWu-AS, where k is the thermal conductivity of the 
medium. (The minus sign indicates that heat flows from hot regions to cold.) Thus the 
rate at which heat flows across the whole surface of the region is given by 


|fcavin-as = -«|| Vu-ds 


Using Gauss’s theorem, we deduce that the rate at which heat flows out of the region is 


-«|]| vvar 


V 


If there are no sources or sinks of heat within the region, this must equal the rate at which 
the region loses heat, -dQ/dt. Therefore 


-¢ |[] cous thdV «|| VoudV 


V Vv 


Since 


this implies that 


|| (er cp) av = 0 


Vv 


This models the situation for any arbitrarily chosen region V. The arbitrariness in the 
choice of V implies that the value of the integral is independent of V and that the 
integrand is equal to zero. Thus 


2, ob du 
Vuz= OL 


The quantity k/cp is termed the thermal diffusivity of the medium and is usually 
denoted by the Greek letter kappa, «. The differential equation models heat flow within 
a medium. Its solution depends on the initial temperature distribution u(r, 0) and on 
the conditions pertaining at the boundary of the region. Methods for solving this equa- 
tion are discussed in Chapter 9. This differential equation also occurs as a model for 
water percolation through a dam, for neutron transport in reactors and in charge transfer 
within charge-coupled devices. We shall now proceed to obtain its solution in a very 
special case. 
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Example 3.34 


Solution 


A large slab of material has an initial temperature distribution such that one half is at 
—uy and the other at +2). Obtain a mathematical model for this situation and solve it, 
stating explicitly the assumptions that are made. 


When a problem is stated in such vague terms, it is difficult to know what approxima- 
tions and simplifications may be reasonably made. Since we are dealing with heat 
transfer, we know that for an isentropic medium the temperature distribution satisfies 
the equation 


1 Ou 
Vu=-2 
. K ot 


throughout the medium. We know that the region we are studying is divided so that at 
t = 0 the temperature in one part is —u) while that in the other is +z, as illustrated in 
Figure 3.50. We can deduce from this figure that the subsequent temperature at a point 
in the medium depends only on the perpendicular distance of the point from the 
dividing plane. We choose a coordinate system so that its origin lies on the dividing 
plane and the x axis is perpendicular to it, as shown in Figure 3.51. Then the differential 
equation simplifies, since u(r, f) is independent of y and z, and we have 


—Uy (x <0) 
+uy (x 2 0) 


Fu _1du 


Pr ; with u(x, 0) = 
x kK ot 


u(r, 0) = u(x, 0) 


= —-u9 


Figure 3.50 Region for Example 3.34. Figure 3.51 Coordinate system for Example 3.34. 


Thinking about the physical problem also provides us with some further information. 
The heat flows from the hot region to the cold until (eventually) the temperature is 
uniform throughout the medium. In this case that terminal temperature is zero, 
since initially half the medium is at temperature +u,) and the other half at —u). So we 
know that u(x, t) > 0 as t > ee. We also deduce from the initial temperature distribution 
that —u) < u(x, t) S wu, for all x and f¢, since there are no extra sources or sinks of heat 
in the medium. Summarizing, we have 


—Uy (x <0) 


+u, (x = 0) 


: Hs.0)=| 
Fu _ 1 du 


ae (-0 <x <0,t 20) with 
x kK ot 


u(x,t) bounded for all x 
u(x,t) 0 as tooo 
There are many approaches to solving this problem (see Chapter 9). One is to investig- 


ate the effect of changing the scale of the independent variables x and t. Setting x = AX 
and t = UT, where A and yl are positive constants, the problem becomes 
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FU _# OU 
Ox’ « OT 
with U(X, T) = u(x, f) and U(X, 0) = uy sgn_X. Choosing 1 = 7’, we see that 
JU =1OU wath 01000) =i sen ¥ 
Ox T 


which implies that the solution u(x, ¢) of the original equation is also a solution of the 
scaled equation. Thus 


u(x, t) = u(Ax, A*t) 


which suggests that we should look for a solution expressed in terms of a new variable 
s that is proportional to the ratio of x to /t. Setting s = ax/\t, we seek a solution as a 
function of s: 


u(x, t) = uofls) 


This reduces the partial differential equation for wu to an ordinary differential equation 
for f, since 


du _dujdf Fu _@ud'f du __1Laxw df 
ox Jt ds ax t ds’ ot 2 tyt ds 
Thus the differential equation is transformed into 
ad'f___ax df 
t ds” 2K tt ds 
giving 
fii 2. 2 at 
ds 2 ds 
Choosing the constant a such that a’ = 1/(4k) reduces this to the equation 
2 
f__»,4f 
ds* ds 


The initial condition is transformed into two conditions, since for x < 0, s 4 —°° as 
t > 0 and for x > 0, s 4 +e as t > 0. So we have 


f(s) 9 1 as soo 
f(s) 2-1 as s—-co 


Integrating the differential equation once gives 
ot A er. where A is a constant 
S 


and integrating a second time gives 


f(s)=B+A | e” ds 
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The integral occurring here is one that frequently arises in heat transfer problems, and 
is given a special name. We define the error function, erf(x), by the integral 


erf(x) = 2| e* dz 


\@ J 9 


Its name derives from the fact that it is associated with the normal distribution, which 

is acommon model for the distribution of experimental errors (see Section 11.2.4). 

This is a well-tabulated function, and has the property that erf(x) > 1 asx > ~, 
Writing the solution obtained above in terms of the error function, we have 


f(s) =A erf(s) +B 
Letting s + co and s + —co gives two equations for A and B: 
1=A+B 
-1=-A4+B 
from which we deduce A = | and B = 0. Thus 
f(s) = erf(s) 
so that 


x/2 \t 
vos-= myer) = 2 | oar 


Show that u(x, y) =x"f(t), t= y/x, satisfies the 
differential equations 


(a) Pk ays nu 
(b) pee + EE a n(n - 1)u 


Verify these results for the function 
u(x, y) =x* +34 + 16x*y”. 


Find the values of the numbers a and b such that 
the change of variables u =x + ay, v =x + by 
transforms the differential equation 


pFf _g Hf +29f 29 
Ox Oxdy oy 
into 
of 
Ouodv 


Hence deduce that the general solution of the 
equation is given by 


u(x, y) =f(x + 3y) + g(x +3y) 


where fand g are arbitrary functions. 
Find the solution of the differential equation 
that satisfies the conditions 


u(x, 0) = sin x, du(x, 0) _ 3 cosx 


A differential P(x, y, z) dx + O(, y, z) dy + 
R(x, y, z) dz is exact if there is a function 
F(, y, Z) such that 


P(x, y, z) dx + O(X, y, z) dy + RG, y, z) dz 
= Vf-(dx, dy, dz) 


Show that this implies V x (P, QO, R) = 0. Deduce 
that curl grad f= 0. 
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Find grad f, plot some level curves f= constant 
and indicate grad f by arrows at some points 
on the level curves for f(r) given by 


(a) xy (b) x/(Qx? + y’) 


Show that if @ is a constant vector then 
(a) grad(@-r)=@ 


(b) curl (@xr)=2@ 


(a) Prove that if f(r) is a scalar point function then 
curl grad f= 0 


(b) Prove that if v = grad [zf(r)] + af(r)k and 
V’f= 0, where @ is a constant and fis a 
scalar point function, then 


divv =(2+ aol V’v = grad (ag ) 
Zz iB 


Show that if F = (x? - y?+x)i- (2xy + y)j, 
then curl F = 0, and find f(r) such that 
F = grad f. 

Verify that 


(2,1) 
| F-dr=[fnley 


(1,2) 


A force F acts on a particle that is moving 
in two dimensions along the semicircle 
x=1-cosé,y=sind(0< 07). 

Find the work done when 


(a) F=\(x°+y)i 
(b) F=\(@?+y)A 


fi being the unit vector tangential to the path. 


A force F = (xy, —y, 1) acts ona particle as it moves 
along the straight line from (0, 0, 0) to (1, 1, 1). 
Calculate the work done. 


The force F per unit length of a conducting wire 
carrying a current J in a magnetic field B is 
F =I B. Find the force acting on a circuit 
whose shape is given by x = sin 8, y=cos 0, 
z=sin ; 6, when current J flows in it and when 
it lies in a magnetic field B=xi-yj+k. 


The velocity v at the point (x, y) ina 
two-dimensional fluid flow is given by 
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16 
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18 
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v = (yi -xj)/(x? + y’). Find the net 
circulation around the square x = +1, y=+1. 


A metal plate has its boundary defined by 
x =0, y=x’/c and y =c. The density at the 
point (x, y) is kxy (per unit area). Find the 
moment of inertia of the plate about an axis 
through (0, 0) and perpendicular to the plate. 


A right circular cone of height / and base radius 
a is cut into two pieces along a plane parallel to 
and distance c from the axis of the cone. Find the 
volume of the smaller piece. 


The axes of two circular cylinders of radius a 
intersect at right angles. Show that the volume 
common to both cylinders may be expressed as 
the triple integral 


: \(a?-y?) \(a?-y?) 
8] dy dx dz 
0 0 0 


and hence evaluate it. 


The elastic energy of a volume V of material 
is ¢-Vi(2EI), where q is its stress and E and J 
are constants. Find the elastic energy of a 
cylindrical volume of radius r and length / in 
which the stress varies directly as the distance 
from its axis, being zero at the axis and q at the 
outer surface. 


The velocity of a fluid at the point (x, y, z) has 
components (3xy, xy’, 0). Find the flow rate out 
of the triangular prism bounded by z = 0, z= 1, 
x=0,y=Oandx+y=1. 


An electrostatic field has components 
(2xy, —y?, x + y) at the point (x, y, z). Find the total 
flux out of the sphere x7 + y?+z° =a’. 


Verify Stokes’ theorem 


Par = \| (curl F)- dS 


(os Ss 
where F = (x? + y — 4, 3xy, 2xz +z’) and S is 
the surface of the hemisphere x? + y* +z? = 16 
above the (x, y) plane. 


Use the divergence theorem to evaluate the 
surface integral 
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20 


|f eas 


Ss 


Zl 


where a = xi + yj — 2zk and S is the surface of 


the sphere x? + y* + z* = a above the (x, y) 
plane. 


Evaluate the volume integral 


|| sear 


V 
where V denotes the wedge-shaped region 
bounded in the positive octant by the four 
planes x = 0, y=0, y=1 —x andz=2 —x. 


Continuing the analysis of Section 3.5, show that 
the net circulation of fluid around the rectangular 
element shown in Figure 3.47 is given by 


[u(x, y + Ay) — u(x, y)JAx 

— [v@ + Ax, y) — vy, y)JAy 
Deduce that if the fluid motion is irrotational at 
(6% y)s then 

du_v_ 4 

Oy ox 
Show that for irrotational incompressible flow, 
the stream function y satisfies Laplace equation 

ay als ay —a@) 

ox oy’ 
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4.1 


Introduction 


In the theory of alternating currents, the application of quantities such as the complex 
impedance involves functions having complex numbers as independent variables. There 
are many other areas in engineering where this is the case; for example, the motion of 
fluids, the transfer of heat or the processing of signals. Some of these applications are 
discussed later in this book. 

Traditionally, complex variable techniques have been important, and extensively used, 
in a wide variety of engineering situations. This has been especially the case in areas 
such as electromagnetic and electrostatic field theory, fluid dynamics, aerodynamics 
and elasticity. With the development of computer technology and the consequential 
use of sophisticated algorithms for analysis and design in engineering there has, over 
the last two decades or so, been less emphasis on the use of complex variable tech- 
niques and a shift towards numerical techniques applied directly to the underlying full 
partial differential equations model of the situation being investigated. However, even 
when this is the case there is still considerable merit in having an analytical solution, 
possibly for an idealized model, in order both to develop better understanding of 
the behaviour of the solution and to give confidence in the numerical estimates for the 
solution of enhanced models. Many sophisticated software packages now exist, many 
of which are available as freeware, downloadable from various internet sites. The older 
packages such as FLUENT and CFX are still available and still in use by engineering 
companies to solve problems such as fluid flow and heat transfer in real situations. The 
finite element package TELEMAC is modular in style and is useful for larger-scale 
environmental problems; these types of software programs use a core plus optional 
add-ons tailored for specific applications. The best use of all such software still requires 
knowledge of mappings and use of complex variables. One should also mention the 
computer entertainment industry which makes use of such mathematics to enable 
accurate simulation of real life. The kind of mappings that used to be used extensively 
in aerodynamics are now used in the computer games industry. In particular the ability 
to analyse complicated flow patterns by mapping from a simple geometry to a complex 
one and back again remains very important. Examples at the end of the chapter illus- 
trate the techniques that have been introduced. Many engineering mathematics texts 
have introduced programming segments that help the reader to use packages such as 
MATLAB or MAPLE to carry out the technicalities. This has not been done in this 
chapter since, in the latest version of MAPLE, the user simply opens the program 
and uses the menu to click on the application required (in this chapter a derivative or 
an integral), types in the problem and presses return to get the answer. Students are 
encouraged to use such software to solve any of the problems; the understanding of 
what the solutions mean is always more important than any tricks used to solve what 
are idealized problems. 

Throughout engineering, transforms in one form or another play a major role in anal- 
ysis and design. An area of continuing importance is the use of Laplace, z, Fourier and 
other transforms in areas such as control, communication and signal processing. Such 
transforms are considered later in the book where it will be seen that complex variables 
play a key role. This chapter is devoted to developing understanding of the standard 
techniques of complex variables so as to enable the reader to apply them with confidence 
in application areas. 
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Figure 4.1 Real 
mapping y = f(x). 


Figure 4.2 Complex 
mapping w = f(z). 
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Complex functions and mappings 


The concept of a function involves two sets X and Y and a rule that assigns to each 
element x in the set X (written x € X) precisely one element y € Y. Whenever this 
situation arises, we say that there is a function / that maps the set_X to the set Y, and 
represent this symbolically by 


y=f(x) eX) 


Schematically we illustrate a function as in Figure 4.1. While x can take any value in 
the set X, the variable y = f(x) depends on the particular element chosen for x. We therefore 
refer to x as the independent variable and y as the dependent variable. The set X is 
called the domain of the function, and the set of all images y = f(x) (x € X) is called 
the image set or range of f: Previously we were concerned with real functions, so that 
x and y were real numbers. If the independent variable is a complex variable z = x + jy, 
where x and y are real and j = (—1), then the function f(z) of z will in general also be 
complex. For example, if f(z) = z’ then, replacing z by x + jy and expanding, we have 


f= +p =O? -y’)+j2xy=u+tjv (say) 
where uw and v are real. Such a function f(z) is called a complex function, and we write 


w=f@) 


where, in general, the dependent variable w = u + ju is also complex. 

The reader will recall that a complex number z = x + jy can be represented on a plane 
called the Argand diagram, as illustrated in Figure 4.2(a). However, we cannot plot 
the values of x, y and f(z) on one set of axes, as we were able to do for real functions 
y =f(x). We therefore represent the values of 


w=f(z)=utjv 


on a second plane as illustrated in Figure 4.2(b). The plane containing the independent 
variable z is called the z plane and the plane containing the dependent variable w is 
called the w plane. Thus the complex function w = f(z) may be regarded as a mapping 
or transformation of points P within a region in the z plane (called the domain) to 
corresponding image points P’ within a region in the w plane (called the range). 

It is this facility for mapping that gives the theory of complex functions much of its 
application in engineering. In most useful mappings the entire z plane is mapped onto 
the entire w plane, except perhaps for isolated points. Throughout this chapter the 
domain will be taken to be the entire z plane (that is, the set of all complex numbers, 
denoted by C). This is analogous, for real functions, to the domain being the entire real 


w=f@) 
i 


Mapping or 
transformation 


Up 


Domain 


zee 
Ly. 


xe 
L 


(a) z plane 


(b) w plane 
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Example 4.1 


Solution 


Figure 4.3 
The mapping of 
Example 4.1. 


line (that is, the set of all real numbers R). If this is not the case then the complex 
function is termed ‘not well defined’. In contrast, as for real functions, the range of the 
complex function may well be a proper subset of C. 


Find the image in the w plane of the straight line y = 2x + 4 in the z plane, z = x + jy, 
under the mapping 


w=27+6 


Writing w = u + jv, where u and v are real, the mapping becomes 
w=utjv=2(x+jy)+6 
or 
utjv= (2x + 6)+j2y 
Equating real and imaginary parts then gives 
u=2x+6, voy (4.1) 
which, on solving for x and y, leads to 
x= (u-6), y= hv 
Thus the image of the straight line 
y=2x+4 
in the z plane is represented by 
Sv=2xi(u-6)+4 
or 
v=2u-—4 


which corresponds to a straight line in the w plane. The given line in the z plane and the 
mapped image line in the w plane are illustrated in Figures 4.3(a) and (b) respectively. 

Note from (1.1) that, in particular, the point P,(—2 + j0) in the z plane is mapped to 
the point P/(2 + j0) in the w plane, and that the point P,(0 + j4) in the z plane is mapped 
to the point P3(6 + j8) in the w plane. Thus, as the point P moves from P, to P, along 


(a) z plane (b) w plane 
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the line y = 2x + 4 in the z plane, the mapped point P’ moves from P/ to P} along the 
line v = 2u — 4 in the w plane. It is usual to indicate this with the arrowheads as 
illustrated in Figure 4.3. 


4.2.1 Linear mappings 


The mapping w = 2z + 6 in Example 4.1 is a particular example of a mapping cor- 
responding to the general complex linear function 


w=az+B (4.2) 
where w and z are complex-valued variables, and a and # are complex constants. In this 
section we shall investigate mappings of the z plane onto the w plane corresponding to 


(4.2) for different choices of the constants a and B. In so doing we shall also introduce 
some general properties of mappings. 


Case (a) a=0 
Letting @= 0 (or a= 0 + j0) in (4.2) gives 
w=B8 


which implies that w = B, no matter what the value of z. This is quite obviously a 
degenerate mapping, with the entire z plane being mapped onto the one point w = B 
in the w plane. If nothing else, this illustrates the point made earlier in this section, 
that the image set may only be part of the entire w plane. In this particular case the 
image set is a single point. Since the whole of the z plane maps onto w = , it follows 
that, in particular, z = B maps to w = PB. The point f is thus a fixed point in this 
mapping, which is a useful concept in helping us to understand a particular mapping. 
A further question of interest when considering mappings is that of whether, given a 
point in the w plane, we can tell from which point in the z plane it came under the 
mapping. If it is possible to get back to a unique point in the z plane then it is said to 
have an inverse mapping. Clearly, for an inverse mapping z = g(w) to exist, the point 
in the w plane has to be in the image set of the original mapping w = f(z). Also, from 
the definition of a mapping, each point w in the w plane image set must lead to a single 
point z in the z plane under the inverse mapping z = g(w). (Note the similarity to the 
requirements for the existence of an inverse function f(x) of a real function f(x).) For 
the particular mapping w = B considered here the image set is the single point w = f in 
the w plane, and it is clear from Figure 4.4 that there is no way of getting back to just 
a single point in the z plane. Thus the mapping w = B has no inverse. 


Figure 4.4 Mapping w = B 
The degenerate z plane 
mapping w = p. 


z plane w plane 
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Figure 4.5 
The mapping 
w=(1+)j)z. 


Case (b) B=0, a#0 
With such a choice for the constants ~ and 3, the mapping corresponding to (4.2) becomes 
w= az 


Under this mapping, the origin is the only fixed point, there being no other fixed points 
that are finite. Also, in this case there exists an inverse mapping 


1 


Z=—w 
a 


that enables us to return from the w plane to the z plane to the very same point 
from which we started under w = az. To illustrate this mapping at work, let us choose 
a=1+4j, so that 


w=(1+j)z (4.3) 


and consider what happens to a general point z,) in the z plane under this mapping. In 
general, there are two ways of doing this. We can proceed as in Example 4.1 and split 
both z and w into real and imaginary parts, equate real and imaginary parts and hence 
find the image curves in the w plane to specific curves (usually the lines Re(z) = con- 
stant, Im(z) = constant) in the z plane. Alternatively, we can rearrange the expression 
for w and deduce the properties of the mapping directly. The former course of action, 
as we shall see in this chapter, is the one most frequently used. Here, however, we shall 
take the latter approach and write @ = 1 +j in polar form as 


1+j=/2e'"" 
Then, if 
z=rel® 
in polar form it follows from (4.3) that 
w= r\20iorn"9 (4.4) 


We can then readily deduce from (4.4) what the mapping does. The general point in the 
z plane with modulus 7 and argument @ is mapped onto an image point w, with modulus 
r/2 and argument 0+ it in the w plane as illustrated in Figure 4.5. 

It follows that in general the mapping 


w= az 


maps the origin in the z plane to the origin in the w plane (fixed point), but effects an expan- 
sion by |@| and an anticlockwise rotation by arg a. Of course, arg a need not be positive, 
and indeed it could even be zero (corresponding to being real). The mapping can be loosely 
summed up in the phrase ‘magnification and rotation, but no translation’. Certain geometrical 


y=Im(z) v=Im (w) 
w 
w= (1+ 4)z 
ee ry2 
; Z 
0 O+4n 
0 x = Re (z) 12) u=Re(w) 
z plane w plane 
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Figure 4.6 
The mapping 
w=C+8. 


properties are also preserved, the most important being that straight lines in the z plane 
will be transformed to straight lines in the w plane. This is readily confirmed by noting 
that the equation of any straight line in the z plane can always be written in the form 


|z—a|=|z—)| 


where a and b are complex constants (this being the equation of the perpendicular 
bisector of the join of the two points representing a and b on the Argand diagram). 
Under the mapping w = az, the equation maps to 


or 
|w-—aa|=|w-ba| 


in the w plane, which is clearly another straight line. 
We now return to the general linear mapping (4.2) and rewrite it in the form 


w—-B=az 
This can be looked upon as two successive mappings: first, 
C= az 


identical to w = az considered earlier, but this time mapping points from the z plane to 
points in the € plane; secondly, 


w=C+ 8 (4.5) 
mapping points in the € plane to points in the w plane. Elimination of ¢ regains equation 
(4.2). The mapping (4.5) represents a translation in which the origin in the ¢ plane is 
mapped to the point w = f in the w plane, and the mapping of any other point in the 
€ plane is obtained by adding f to the coordinates to obtain the equivalent point in the 
w plane. Geometrically, the mapping (4.5) is as if the € plane is picked up and, without 
rotation, the origin placed over the point f. The original axes then represent the w plane 
as illustrated in Figure 4.6. Obviously al/ curves, in particular straight lines, are pre- 
served under this translation. 

We are now in a position to interpret (4.2), the general linear mapping, geometrically 
as a combination of mappings that can be regarded as fundamental, namely 


e translation 
e rotation, and 
e magnification 


that is, 
j0 jd iO 

——> e”z ——_> | ae’ —>| ale"z+ Be az+B=w 

rotation magnification translation 
w=l+p 

_ 
op 
o Q 
Cplane, = €) + JO w plane, w = u + jv 
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Example 4.2 


Solution 


It clearly follows that a straight line in the z plane is mapped onto a corresponding 
straight line in the w plane under the linear mapping w = az + f. A second useful 
property of the linear mapping is that circles are mapped onto circles. To confirm this, 
consider the general circle 


|z-—z|=r 


in the z plane, having the complex number Zz, as its centre and the real number + as its 
radius. Rearranging the mapping equation w = az + B gives 


where wy = z, + B. Hence 
|z-—Zl=r 

implies 
|w — wo| =|a|r 


which is a circle, with centre wy given by the image of z, in the w plane and with radius 
|a|r given by the radius of the z plane circle magnified by |]. 
We conclude this section by considering examples of linear mappings. 


Examine the mapping 
w=(1+j)z+1-j 


as a succession of fundamental mappings: translation, rotation and magnification. 


The linear mapping can be regarded as the following sequence of simple mappings: 


jn/4 jn/4 


\ / jn/4 : 

——> |2e""'z —__> |2e""z+1-j=w 
rotation magnification translation 

anticlockwise by \2 0 1-j or 


by in (0,0) >(1,-1) 


Figure 4.7 illustrates this process diagrammatically. The shading in Figure 4.7 helps to 
identify how the z plane moves, turns and expands under this mapping. For example, 
the line joining the points 0 + j2 and | + j0 in the z plane has the cartesian equation 


tytx= 1 

Taking w=u+juv and z= x + jy, the mapping 
w=(1+j)z+1-j 

becomes 


utjv=(14+jpa@tjy+1—-j=@-yt+tlt+jat+y-1) 
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Figure 4.7 
The mapping 
w=(1+j)z+1-j. 


Example 4.3 
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: 4 


V2eit/4z + | = j 
— 


| 
ei 
Dl 


Equating real and imaginary parts then gives 


liz 
ins 
lr 
} 

Z_] 


u=x-ytl, v=xty-1 
which on solving for x and y gives 
2x=uty, 2y=v-—u+2 


Substituting for x and y into the equation }y + x = 1 then gives the image of this line in 
the w plane as the line 


3v+u=2 


which crosses the real axis in the w plane at 2 and the imaginary axis at A . Both lines 
are shown dashed, in the z and w planes respectively, in Figure 4.7. 


The mapping w = az + B (a, B constant complex numbers) maps the point z = 1 + j 
to the point w =j, and the point z = 1 —j to the point w=-1. 


(a) Determine a and PB. 


(b) Find the region in the w plane corresponding to the right half-plane Re(z) = 0 
in the z plane. 


(c) Find the region in the w plane corresponding to the interior of the unit circle 
|z| < 1 in the z plane. 


(d) Find the fixed point(s) of the mapping. 


In (b)—(d) use the values of @ and B determined in (a). 
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Solution 


(a) 


(b) 


The two values of z and w given as corresponding under the given linear mapping 
provide two equations for a and B as follows: z = 1 + j mapping to w =j 
implies 

j=a(l+j+B 

while z = | — j mapping to w = —1 implies 

-l=a(l1-j)+fB 

Subtracting these two equations in @ and B gives 

j+l=ad+j)-ad -j) 

so that 

a= St =11-)) 

Substituting back for B then gives 

B=j-(+jja=j-3U-p)=j-1 

so that 

w= 3 —-je+j-1=(1-)Gz-1) 


The best way to find specific image curves in the w plane is first to express 
z(=x+Jy) in terms of w (=u + jv) and then, by equating real and imaginary parts, 
to express x and y in terms of u and v. We have 


w=(1-j)Gz- 1) 
which, on dividing by | —j, gives 


Ww 
re wee 


Taking w=u+juv and z=x + jy and then rationalizing the left-hand side, we have 
3(u + jolt j= 3@+jy)-1 

Equating real and imaginary parts then gives 

u-v=x-2, utv=y (4.6) 


The first of these can be used to find the image of x = 0. It is uw -— v = —2, which 
is also a region bordered by the straight line w — v = —2 and shown in Figure 4.8. 
Pick one point in the right half of the z plane, say z = 2, and the mapping gives 
w = 0 as the image of this point. This allays any doubts about which side of 
u—v=-2 corresponds to the right half of the z plane, x => 0. The two correspond- 
ing regions are shown ‘hatched’ in Figure 4.8. 


Note that the following is always true, although we shall not prove it here. If a 


curve cuts the z plane in two then the corresponding curve in the w plane also cuts 
the w plane in two, and, further, points in one of the two distinct sets of the z plane 
partitioned by the curve correspond to points in just one of the similarly partitioned 
sets in the w plane. 
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The mappings of 
Example 4.3. 
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(c) 


(d) 


z plane w plane 


In cartesian form, with z = x + jy, the equation of the unit circle |z| = 1 is 
r+ye=l 


Substituting for x and y from the mapping relationships (4.6) gives the image of 
this circle as 


(u—v+2~P+(ut+vy=1 

or 

w+ +2u—2v+ 3 =0 

which, on completing the squares, leads to 
(u+ 1P+(v-1P =4 


As expected, this is a circle, having in this particular case centre (-1, 1) and 
radius +. Ifx?+y? < 1 then (w+ 1)’ + (v— 1) < }, so the region inside the 
circle |z| = 1 in the z plane corresponds to the region inside its image circle in 
the w plane. Corresponding regions are shown shaded in Figure 4.8. 


The fixed point(s) of the mapping are obtained by putting w =z in w= az + B, 
leading to 


z=(5z- 1)(1-j) 
that is, 


z=}z-$jz-1+j 


so that 
aera 
2+ 3] 


is the only fixed point. 


One final point is in order before we leave this example. In Figure 4.8 the images of 
x =0 and x° + y = 1 can also be seen in the context of translation, rotation (the line in 


Figure 4.8 rotates about z = 2j) and magnification (in fact, shrinkage, as can be seen by 
the decrease in diameter of the circle compared with its image in the w plane). 


www.2Ofile.org 


268 


FUNCTIONS OF A COMPLEX VARIABLE 


4.2.2 Exercises 


Find in the cartesian form y = mx + c (m and c real 
constants) the equations of the following straight 
lines in the z plane, z = x + jy: 


(a) |z—2+j]=|z-j+3]| 
(b) z+2z* + 4j(z-z*) =6 


where * denotes the complex conjugate. 


Find the point of intersection and the angle of 
intersection of the straight lines 


le=L=j[=l2=34]l| 
Iz—1+j|=lz-3-j| 


The function w = jz + 4 — 3j is a combination of 
translation and rotation. Show this diagrammatically, 
following the procedure used in Example 4.2. Find 
the image of the line 6x + y = 22 (z =x + jy) in the 
w plane under this mapping. 


Show that the mapping w = (1 — j)z, where 
w=u+jvandz=x+J]y, maps the region y > 1 
in the z plane onto the region w+ v > 2 in the 

w plane. Illustrate the regions in a diagram. 


Under the mapping w = jz+j, where w=u+jv 
and z = x + jy, show that the half-plane x > 0 
in the z plane maps onto the half-plane v > 1 in the 
w plane. 


4.2.3 Inversion 


The inversion mapping is of the form 


All| 
w=- 
Z 


For z =x + jy find the image region in the w plane 
corresponding to the semi-infinite strip x > 0, 

0 <y <2 in the z plane under the mapping 

w = jz + 1. Illustrate the regions in both planes. 


Find the images of the following curves under 
the mapping 


w=(jt /3)z+j/3-1 
(a) y=0 


(c) P+y=1 


(b) x=0 
(d) x? +y4+2y=1 


where z =x +]y. 


The mapping w = az + B (a, B both constant 
complex numbers) maps the point z = 1 + j to 
the point w =j and the point z = —1 to the point 
w=1+4)j. 


(a) Determine @ and f. 

(b) Find the region in the w plane 
corresponding to the upper half-plane 
Im(z) > 0 and illustrate diagrammatically. 

(c) Find the region in the w plane corresponding to 
the disc |z| < 2 and illustrate diagrammatically. 

(d) Find the fixed point(s) of the mapping. 


In (b)-(d) use the values of @ and B determined 
in (a). 


(4.7) 


and in this subsection we shall consider the image of circles and straight lines in the 
z plane under such a mapping. Clearly, under this mapping the image in the w plane of 


the general circle 


|z-—Zl=r 


in the z plane, with centre at z,) and radius r, is given by 


(4.8) 


but it is not immediately obvious what shaped curve this represents in the w plane. To 
investigate, we take w =u + jv and z) =X) + jy in (4.8), giving 
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u— jv F 
7 3 *0 ~ J¥o 
uit+v 


=fr 


Squaring we have 


2 2 
u v 2 
( a ;~ %) +( a s+ yo) =F 
urU ut+v 


which on expanding leads to 


ue 2uX9 2 v 2vVo 2 2 
2.22 2, 2 0b WZ 95 2, 2 = 
(ut+v) uot+v (ui +vu') (u+v) 
or 
2,.2 
u_tv 2vyo — 2UX) _ 2 2 
2 2,2 2 2 =P *Xo~ Yo 
(u'+v) uotv 
so that 
(w+ v°\(r? — x6 — yp) + 2uxy — 2vy = 1 (4.9) 


The expression is a quadratic in u and v, with the coefficients of uv’ and v? equal and no 
term in uv. It therefore represents a circle, unless the coefficient of u’ + v” is itself zero, 
which occurs when 

xetyo=r’, or |Zl=r 
and we have 

2uxXy — 2vy, = | 


which represents a straight line in the w plane. 


Summarizing, the inversion mapping w = 1/z maps the circle |z — z)| = in the z 
plane onto another circle in the w plane unless |z)| = 7, in which case the circle is 
mapped onto a straight line in the w plane that does not pass through the origin. 


When |z)| # 7, we can divide the equation of the circle (4.9) in the w plane by the 
factor r? — xj — yo to give 


2X yu > 2Vov _ 1 


2 2, 


2,2 
ee el 2 2 2 D2 2 2 
roe—~Xo7— Vo 7 Xo ~ Vo FT ~ Xo — Vo 


which can be written in the form 
(u — Uy)” +(v—v) =R 


where (up, V9) are the coordinates of the centre and R the radius of the w plane circle. It 
is left as an exercise for the reader to show that 


Xx 
(Uo, Vo) = (-<=*. —*—), R=>—— 
r-|zZo/° r - [20 r —|Zo| 


Next we consider the general straight line 


|z—a,|=|z—-a,| 
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in the z plane, where a, and a, are constant complex numbers with a, # a,. Under the 
mapping (4.7), this becomes the curve in the w plane represented by the equation 


(4.10) 


Again, it is not easy to identify this curve, so we proceed as before and take 
w=utjpv, a,=pt+jq, a,=r+js 


where p, g, rand s are real constants. Substituting in (4.10) and squaring both sides, we 


have 
2 2 2 2 
u v 7 u : v 
(= ;-p) +(s sta) =(s sr} +(5 5+5) 
ut+ov uotov ut+ov ut+ov 


Expanding out each term, the squares of u/(w? + v*) and v/(u’ + v’) cancel, giving 


_ _2up + p+ ee 2ur 4 ge 2vs as 


w+ utv utv uty 
which on rearrangement becomes 
(wt+v)(p?+¢ —1r —s’) + 2u(r—- p) + 20(q - 5) =0 (4.11) 
Again this represents a circle through the origin in the w plane, unless 
pt+g=rt+s 


which implies | a,| =|a@,|, when it represents a straight line, also through the origin, in 
the w plane. The algebraic form of the coordinates of the centre of the circle and its 
radius can be deduced from (4.11). 


We can therefore make the important conclusion that the inversion mapping 
w = I/z takes circles or straight lines in the z plane onto circles or straight lines in 
the w plane. Further, since we have carried out the algebra, we can be more 
specific. If the circle in the z plane passes through the origin (that is, | z)| =7 in (4.9) ) 
then it is mapped onto a straight line that does not pass through the origin in the w 
plane. If the straight line in the z plane passes through the origin (|a@,| = |a5| in 
(4.11)) then it is mapped onto a straight line through the origin in the w plane. 
Figure 4.9 summarizes these conclusions. 


To see why this is the case, we first note that the fixed points of the mapping, deter- 
mined by putting w = z, are 
z=-, or 7=l 
Z 


so that z=+1. 

We also note that z = 0 is mapped to infinity in the w plane and w = 0 is mapped to 
infinity in the z plane and vice versa in both cases. Further, if we apply the mapping a 
second time, we get the identity mapping. That is, if 


woe, and —— 
Zz w 
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Figure 4.9 
The inversion 
mapping w = 1/z. 
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z plane Mapping w = a w plane 
y v 
fo N lS 
wy KY 
y v 
es 
O x Oo u 
y v 
OO 
oO x Oo u 
y v 
ie 
oO x 0 u 
then 
We 
G=t =: 
1/z 


which is the identity mapping. 

The inside of the unit circle in the z plane, |z| < 1, is mapped onto |1/w| < 1 or 
|w| > 1, the outside of the unit circle in the w plane. By the same token, therefore, 
the outside of the unit circle in the z plane |z| > | is mapped onto |1/w| > 1 or 
|w| < 1, the inside of the unit circle in the w plane. Points actually on |z| = | in the 
z plane are mapped to points on |w| = | in the w plane, with +1 staying fixed, as 
already shown. Figure 4.10 summarizes this property. 

It is left as an exercise for the reader to show that the top half-boundary of |z| = 1 is 
mapped onto the bottom half-boundary of | w| = 1. 

For any point z, in the z plane the point 1/z, is called the inverse of z, with respect 
to the circle |z| = 1; this is the reason for the name of the mapping. (Note the double 
meaning of inverse; here it means the reciprocal function and not the ‘reverse’ 
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Figure 4.10 Mapping 
of the unit circle under 
w= l/z. 


Example 4.4 


Solution 


mapping.) The more general definition of inverse is that for any point z, in the z plane 
the point r’/z, is the inverse of z, with respect to the circle |z| = r, where r is a real 
constant. 


Determine the image path in the w plane corresponding to the circle |z — 3| = 2 in the 
z plane under the mapping w = 1/z. Sketch the paths in both the z and w planes and 
shade the region in the w plane corresponding to the region inside the circle in the 
z plane. 


The image in the w plane of the circle |z — 3| = 2 in the z plane under the mapping 
w = I/z is given by 


-3|=2 


which, on taking w = w+ jv, gives 


sore la 


2 2 
uo+v 


Squaring both sides, we then have 
2 _ 2 
Gt T(ae) 
ut+v uo+v 


ee Oe enh 
2 <9 2 2 - 
(uw +v) uo+v 


or 


which reduces to 
1-6u+5(w+v°)=0 
or 
(u-?P+v=s 
2 


Thus the image in the w plane is a circle with centre (3, 0) and radius ?. The cor- 
responding circles in the z and w planes are shown in Figure 4.11. 
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Figure 4.11 
The mapping of 
Example 4.4. 


4.2.4 
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YA 1 ae 
Wr 
— 
B’ 
A G A 
oe {a 
O 1 5 x O!C 1 u 
B 
z plane w plane 


Taking z = x + jy, the mapping w = 1/z becomes 


utjv= —_ aoe 
x+jy x+y 
which, on equating real and imaginary parts, gives 


x -y 
cs 2? Oe 2 
x +y x +y 


We can now use these two relationships to determine the images of particular points 
under the mapping. In particular, the centre (3, 0) of the circle in the z plane is mapped 
onto the point v= +, v=0 in the w plane, which is inside the mapped circle. Thus, under 
the mapping, the region inside the circle in the z plane is mapped onto the region inside 
the circle in the w plane. 

Further, considering three sample points A(1 + j0), B(3 — j2) and C(5 + j0) on the 
circle in the z plane, we find that the corresponding image points on the circle in the w 
plane are A’(1, 0), B’(, 4) and C’(£, 0). Thus, as the point z traverses the circle in the 
z plane in an anticlockwise direction, the corresponding point w in the w plane will also 
traverse the mapped circle in an anticlockwise direction as indicated in Figure 4.11. 


Bilinear mappings 
A bilinear mapping is a mapping of the form 


az+b 
= tee 4.12 
e cz +d ( ) 


where a, b, c and d are prescribed complex constants. It is called the bilinear mapping 
in z and w since it can be written in the form Awz + Bw + Cz + D=0, which is linear 
in both z and w. 

Clearly the bilinear mapping (4.12) is more complicated than the linear mapping 
given by (4.2). In fact, the general linear mapping is a special case of the bilinear 
mapping, since setting c = 0 and d= 1 in (4.12) gives (4.2). In order to investigate the 
bilinear mapping, we rewrite the right-hand side of (4.12) as follows: 

a ad 
gh ae 
eztd cz+d 
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so that 


a bc — ad 
af) ee 4.1 
. c - c(cz + d) i 


This mapping clearly degenerates to w = a/c unless we demand that bc — ad # 0. We 
therefore say that (4.12) represents a bilinear mapping provided the determinant 


b 
c ad 


| = ad~ bc 


is non-zero. This is sometimes referred to as the determinant of the mapping. When 
the condition holds, the inverse mapping 


_-dwt+hb 
L£aee———_—_—_— 
cw-a 


obtained by rearranging (4.12), is also bilinear, since 
Pe b 


c -a 


=da-—cb#0 


Renaming the constants so that A = a/c, U = be — ad, a= ¢ and B = cd, (4.13) 
becomes 


w=A+ 4. 
az+ B 


and we can break the mapping down into three steps as follows: 


z,=az+B 
1 
Z== 
21 

w=A+t+UZ 


The first and third of these steps are linear mappings as considered in Section 4.2.1, 
while the second is the inversion mapping considered in Section 4.2.3. The bilinear 
mapping (4.12) can thus be generated from the following elementary mappings: 


1 


wee B inversion Qyz+ B 


: Oz : 
rotation translation 
and 
magnification 


magnification qyz + B translation az+ B 
and 
rotation 

We saw in Section 4.2.1 that the general linear transformation w = az + B does not 
change the shape of the curve being mapped from the z plane onto the w plane. Also, 
in Section 4.2.3 we saw that the inversion mapping w = 1/z maps circles or straight lines 
in the z plane onto circles or straight lines in the w plane. It follows that the bilinear 
mapping also exhibits this important property, in that it also will map circles or straight 
lines in the z plane onto circles or straight lines in the w plane. 
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Solution 
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Investigate the mapping 


graces 
z+l1 


by finding the images in the w plane of the lines Re(z) = constant and Im(z) = constant. 
Find the fixed points of the mapping. 


Since we are seeking specific image curves in the w plane, we first express z in terms 
of w and then express x and y in terms of u and v, where z= x + jy and w=u+ jo. 
Rearranging 


gives 


~ T-w 
Taking z=x+jy and w=u + jv, we have 
evqr= l+utiv 
l-u-jv 


_ L+utjv l-utjv 
l-u-jv l-utjv 


which reduces to 


2 2 
ee nae a 44 2v 


——— 
(l-uyt+v “(-uy+uv 


Equating real and imaginary parts then gives 


_ as 
ss (4.14a) 
(l-u) +v 
2 
y= —— (4.14b) 
(l-u) +v 


It follows from (4.14a) that the lines Re(z) = x = c,, which are parallel to the imaginary 
axis in the z plane, correspond to the curves 


a es 
,= es 
(l-uy+v 


where c, is a constant, in the w plane. Rearranging this leads to 
cl -2u+vt+vry=1l-v-v 
or, assuming that 1 + c, # 0, 


2c\u c,- 1 
w+y-—— + =0 
l+c¢, c, +1 
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Figure 4.12 
The mapping 
w=(z-1)\/(z+ 1). 


which, on completing squares, gives 


2 2 
(aa) lot 
l+ce, l+c, 


It is now clear that the corresponding curve in the w plane is a circle, centre (u = 
c,/( + ¢,), v= 0) and radius (1 + ¢,)". 

In the algebraic manipulation we assumed that c, # —1, in order to divide by 1 + c,. 
In the exceptional case c,; = —1, we have u = 1, and the mapped curve is a straight line 
in the w plane parallel to the imaginary axis. 

Similarly, it follows from (4.14b) that the lines Im(z) = y = c), which are parallel to 
the imaginary axis in the z plane, correspond to the curves 


i 2v 
eae 
(l-u)+v 


where c, is a constant, in the w plane. Again, this usually represents a circle in the w 
plane, but exceptionally will represent a straight line. Rearranging the equation we have 
2v 


C2 


(-uy+ve= 


provided that c, # 0. Completing the square then leads to 
2 
(u- v+(v-4) -1 
C2 


which represents a circle in the w plane, centre (vu = 1, v = 1/c,) and radius 1/c). 

In the exceptional case c, = 0, v= 0 and we see that the real axis y = 0 in the z plane 
maps onto the real axis v = 0 in the w plane. 

Putting a sequence of values to c, and then to c,, say —10 to +10 in steps of +1, 
enables us to sketch the mappings shown in Figure 4.12. The fixed points of the map- 
ping are given by 


www.2Ofile.org 


Example 4.6 


Solution 
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that is, 
2=-l, or z=4j 


In general, all bilinear mappings will have two fixed points. However, although there 
are mathematically interesting properties associated with particular mappings having 
coincident fixed points, they do not impinge on engineering applications, so they only 
deserve passing reference here. 


Find the image in the w plane of the circle |z| = 2 in the z plane under the bilinear 
mapping 


Sl | 
zt+j 


Sketch the curves in both the z and w planes and shade the region in the w plane cor- 
responding to the region inside the circle in the z plane. 


Rearranging the transformation, we have 


so that the image in the w plane of the circle |z| = 2 in the z plane is determined by 


wed =% (4.15) 


One possible way of proceeding now is to put w =u + jv and proceed as in Example 4.4, 
but the algebra becomes a little messy. An alternative approach is to use the property 
of complex numbers that | z,/z,| = |z, |/|z,|, so that (4.15) becomes 


[jwt+jl=2|1-w| 

Taking w = u + jv then gives 
|-v+j(ut+ 1)]=2/C -—u)-jo| 

which on squaring both sides leads to 
yt+ituy=4[ -uyP+v] 

or 
wt But 1=0 

Completing the square of the u term then gives 
(u— 2° +y= % 


indicating that the image curve in the w plane is a circle centre (w= ?, v= 0) and radius 
+. The corresponding circles in the z and w planes are illustrated in Figure 4.13. To 
identify corresponding regions, we consider the mapping of the point z= 0 + j0 
inside the circle in the z plane. Under the given mapping, this maps to the point 
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Figure 4.13 
The mapping 
w=(z-j(z+j). 


Example 4.7 


Solution 


w= 2=1 =-1+50 
0+) 


in the w plane. It then follows that the region inside the circle |z| = 2 in the z plane maps 
onto the region outside the mapped circle in the w plane. 


a 
+> 
O}t 2 u 
z plane w plane 


An interesting property of (4.12) is that there is just one bilinear transformation that 
maps three given distinct points z,, z, and z; in the z plane onto three specified distinct 
points w,, w, and w; respectively in the w plane. It is left as an exercise for the reader 
to show that the bilinear transformation is given by 


(w = W))(W2 = W3) = (Z — 2;)(22 — 23) (4.16) 
(w—w3)(W2-—W)) (2 — 23)(22 — 21) 


The right-hand side of (4.16) is called the cross-ratio of z,, z,, z; and z. We shall illus- 
trate with an example. 


Find the bilinear transformation that maps the three points z = 0, —j and —1 onto the 
three points w = j, 1, 0 respectively in the w plane. 


Taking the transformation to be 


_azt+b 
cz+d 


on using the given information on the three pairs of corresponding points we have 


-_a(0)+b_b 4.17 
I~ Ojeda ia 


p= 2D +4 (4.17b) 
c(-j) +d 
i Ge 4.17 
Ota d ono 
From (4.17c) a = b; then from (4.17a) 


go2 = jh 
j 
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13 


and from (4.17b) c = ja. Thus 


jaz —ja 


— azt+a _lz+l_ 
jz-1 
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zt] 
z-1 


Alternatively, using (4.16) we can obtain 


(w-j)(1- 0) _ @-0)Ci+ 1 
te O)= i) tee (a0) 


or 


as before. 


4.2.5 Exercises 


Show that if z = x + jy, the image of the half-plane 
y > c(c constant) under the mapping w = 1/z is the 
interior of a circle, provided that c > 0. What is 
the image when c = 0 and when c < 0? Illustrate 
with sketches in the w plane. 


Determine the image in the w plane of the circle 


7 


3: a eee 8 
Z+itj =4 


under the inversion mapping w = 1/z. 


Show that the mapping w = 1/z maps the circle 
|z — a|=a, with a being a positive real constant, 
onto a straight line in the w plane. Sketch the 
corresponding curves in the z and w planes, 
indicating the region onto which the interior 

of the circle in the z plane is mapped. 


Find a bilinear mapping that maps z = 0 to w =j, 
z=-jtow=1andz=~-1 to w=0. Hence sketch 
the mapping by finding the images in the w plane 
of the lines Re(z) = constant and Im(z) = constant in 
the z plane. Verify that z = $(j — 1)(-1 £ 3) are 
fixed points of the mapping. 


The two complex variables w and z are related 
through the inverse mapping 


ee) 
Zz 


w 


(a) Find the images of the points z= 1, 1 —j and 
0 in the w plane. 

(b) Find the region of the w plane corresponding 
to the interior of the unit circle |z| < 1 in the 
z plane. 
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15 


16 


AW, 


(c) Find the curves in the w plane corresponding 
to the straight lines x = y and x + y= 1 in the 
z plane. 

(d) Find the fixed points of the mapping. 


Given the complex mapping 


aetl 
z-1 


Ww 


where w =u + jv andz=x+jy, determine the 
image curve in the w plane corresponding to the 
semicircular arc x° + y’ = | (x < 0) described from 
the point (0, —1) to the point (0, 1). 


(a) Map the region in the z plane (z = x + jy) that 
lies between the lines x = y and y= 0, withx < 0, 
onto the w plane under the bilinear mapping 


224 

Z-—3 
(Hint: Consider the point w = 2 to help identify 
corresponding regions.) 

(b) Show that, under the same mapping as in (a), 
the straight line 3x + y = 4 in the z plane 
corresponds to the unit circle | w| = 1 in the 
w plane and that the point w = | does not 
correspond to a finite value of z. 


w= 


If w = (z —j)/(z + j), find and sketch the image in 
the w plane corresponding to the circle |z|=2 in the 
z plane. 


Show that the bilinear mapping 


J09 Z — Zo 
w=e =n ae 
Z—Zo 
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FUNCTIONS OF A COMPLEX VARIABLE 


where 6, is a real constant 0 < @ < 27, z, a fixed 
complex number and z* its conjugate, maps the 
upper half of the z plane (Im(z) > 0) onto the inside 
of the unit circle in the w plane (|w| < 1). Find the 
values of z, and 0, if w = 0 corresponds to z = j and 
w =-1 corresponds to z = ©. 


Show that, under the mapping 


w= Bie. 
Zt+J 


4.2.6 The mapping w = z? 


19 


circular arcs or the straight line through z = 0 and 
z=j in the z plane are mapped onto circular arcs 
or the straight line through w = 0 and w =j in the 
w plane. Find the images of the regions | z — S |< 5 
and |z| < |z—j| in the w plane. 


Find the most general bilinear mapping that maps 
the unit circle |z| = 1 in the z plane onto the unit 
circle | w|= 1 in the w plane and the point z = z, in 
the z plane to the origin w = 0 in the w plane. 


There are a number of other mappings that are used by engineers. For example, in 
dealing with Laplace and z transforms, the subjects of Chapters 5 and 6 respectively, 
we are concerned with the polynomial mapping 


W=a)+azt+...+4a,Z" 


where dp, a, .. 


_ P(z) 
”* Ol) 


. , a, are complex constants, the rational function 


where P and Q are polynomials in z, and the exponential mapping 


w=ae™ 


where e = 2.71828..., the base of natural logarithms. As is clear from the bilinear 
mapping in Section 4.2.4, even elementary mappings can be cumbersome to analyse. 
Fortunately, we have two factors on our side. First, very detailed tracing of specific 
curves and their images is not required, only images of points. Secondly, by using com- 
plex differentiation, the subject of Section 4.3, various facets of these more complicated 
mappings can be understood without lengthy algebra. As a prelude, in this subsection 
we analyse the mapping w = z’, which is the simplest polynomial mapping. 


Investigate the mapping w = z’ by plotting the images on the w plane of the lines 


x = constant and y = constant in the z plane. 


Solution 


There is some difficulty in inverting this mapping to get z as a function of w, since 


square roots lead to problems of uniqueness. However, there is no need to invert here, 
for taking w= u+ jv and z= x + jy, the mapping becomes 


wut jv=(x+jyy =e —y”) + j2xy 


which, on taking real and imaginary parts, gives 


u=x-y 


v = 2xy 


(4.18) 
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Figure 4.14 
The mapping w = 2’. 
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If x = a, a real constant, then (4.18) becomes 
u=-y’, v=2ay 


which, on eliminating y, gives 


u=o-— +. 
4a 
or 
4oru=40 -v° 
so that 


v= 400 — 40°u = 4070 — u) 


This represents a parabola in the w plane, and, since the right-hand side must be 
positive, o° = u so the ‘nose’ of the parabola is at « = a on the positive real axis in 
the w plane. 

If y= B, a real constant, then (4.18) becomes 


u=x — B, v =2xB 


which, on eliminating x, gives 


or 


so that 
v= 48'u + 4B' = 4B’(u + B’) 


This is also a parabola, but pointing in the opposite direction. The right-hand side, as 
before, must be positive, so that wu > —B* and the ‘nose’ of the parabola is on the 
negative real axis. These curves are drawn in Figure 4.14. 


w plane 
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20 


21 


22 


We shall not dwell further on the finer points of the mapping w =z’. Instead, we note 
that in general it is extremely difficult to plot images of curves in the z plane, even the 
straight lines parallel to the axes, under polynomial mappings. We also note that we do 
not often need to do so, and that we have done it only as an aid to understanding. 

The exercises that follow should also help in understanding this topic. We shall then 
return to examine polynomial, rational and exponential mappings in Section 4.3.4, after 
introducing complex differentiation. 


4.2.7 Exercises 


Find the image region in the w plane corresponding 
to the region inside the triangle in the z plane having 
vertices at 0 + j0, 2 +j0 and 0 + j2 under the 
mapping w =z’. Illustrate with sketches. 


Find the images of the lines y = x and y = —x under 
the mapping w =z’. Also find the image of the 
general line through the origin y = mx. By putting 
m = tan @ , deduce that straight lines intersecting at 
the origin in the z plane map onto lines intersecting 
at the origin in the w plane, but that the angle 
between these image lines is double that between 
the original lines. 


Consider the mapping w = z", where 7 is an integer 
(a generalization of the mapping w = z’). Use the 
polar representation of complex numbers to show 
that 


(a) Circles centred at the origin in the z plane are 
mapped onto circles centred at the origin in the 
w plane. 


23) 


Complex differentiation 


(b) Straight lines passing through the origin 
intersecting with angle 0, in the z plane are 
mapped onto straight lines passing through the 
origin in the w plane but intersecting at an 
angle np. 


If the complex function 


is represented by a mapping from the z plane onto 
the w plane, find w in terms of x and y, and v in terms 
of x and y, where z=x+Jjy, w=u + ju. Find the 
image of the unit circle |z| = 1 in the w plane. Show 
that the circle centred at the origin, of radius r, in 
the z plane (|z| = 7) is mapped onto the curve 


Pr 2 ey \2 
(+) +(4 je? (r #1) 
r+1 r—1 


in the w plane. What kind of curves are these? What 
happens for very large r? 


The derivative of a real function f(x) of a single real variable x at x = x) is given by the 


limit 


Sf’ (%) = lim oe 


AKO = XG 


Here, of course, x) is a real number and so can be represented by a single point on the 
real line. The point representing x can then approach the fixed x, either from the left or 
from the right along this line. Let us now turn to complex variables and functions 
depending on them. We know that a plane is required to represent complex numbers, 
SO Zy is now a fixed point in the Argand diagram, somewhere in the plane. The definition 
of the derivative of the function f(z) of the complex variable z at the point z,) will thus be 
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Peis ia [erated 


Z— Zo 


It may appear that if we merely exchange z for x, the rest of this section will follow 
similar lines to the differentiation of functions of real variables. For real variables 
taking the limit could only be done from the left or from the right, and the existence of 
a unique limit was not difficult to establish. For complex variables, however, the point 
that represents the fixed complex number z, can be approached along an infinite num- 
ber of curves in the z plane. The existence of a unique limit is thus a very stringent 
requirement. That most complex functions can be differentiated in the usual way is a 
remarkable property of the complex variable. Since z = x + jy, and x and y can vary 
independently, there are some connections with the calculus of functions of two real 
variables, but we shall not pursue this connection here. 

Rather than use the word ‘differentiable’ to describe complex functions for which a 
derivative exists, if the function f(z) has a derivative f’(z) that exists at all points of a 
region R of the z plane then f(z) is called analytic in R. Other terms such as regular or 
holomorphic are also used as alternatives to analytic. (Strictly, functions that have a 
power series expansion — see Section 4.4.1 — are called analytic functions. Since dif- 
ferentiable functions have a power series expansion they are referred to as analytic 
functions. However, there are examples of analytic functions that are not differentiable.) 


Cauchy-Riemann equations 

The following result is an important property of the analytic function. 
Ifz=x+Jy and f(z) = u(x, y) + ju(x, y), and f(z) is analytic in some region R of the 
z plane, then the two equations 


dud du__ a 
ox oy oy ox 


known as the Cauchy—Riemann equations, hold throughout R. 


(4.19) 


It is instructive to prove this result. Since f’(z) exists at any point z, in R, 


Pes tim [estate 


Z— Zo 
where z can tend to z, along any path within R. Examination of (4.19) suggests that 
we might choose paths parallel to the x direction and parallel to the y direction, since 
these will lead to partial derivatives with respect to x and y. Thus, choosing z — z) = Ax, 
a real path, we see that 


fle) = Yim | feo Ax) = fl20) 
Ax>0 Ax 


Since f(z) = u + jv, this means that 


f'(&) = Jim U(xXo + Ax, Yo) + ju(%o + ae Vo) = U(X0, Yo) = JUM%o, Yo) 
X—> x 
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or, on splitting into real and imaginary parts, 


Cy) = lim | HOO A Yo) = wt. Yo) 4, uC + Ay) = HE. Yo) 
Ax Ax 


giving 


f’@) “|e is (4.20) 
x X=Xq V=Vo 


Starting again from the definition of f’(z,), but this time choosing z — z, = jAy for the 
path parallel to the y axis, we obtain 


f’() = lim | flzo + jAy) = flz0) 
0 ve, jAy 


Once again, using f(z) = u + jv and splitting into real and imaginary parts, we see that 


f'(%) = lim [ u(xo, Yo + Ay) + jv(%o, Yo + Ay) - U(X, Vo) — JV%o» Yo) 
jAy>0 iA 

L JAY 
= hin Lu(xo, Yo + Ay) = U(Xo, Yo) 4 V(Xo, Yo t+ Ay) = V(X, Vo) 
j Ay Ay 


giving 


1 Ou al (4.21) 


f (Zo) = oe oy 
Since f’(z)) must be the same no matter what path is followed, the two values obtained 
in (4.20) and (4.21) must be equal. Hence 


Ou gjet ate ee Ou , ov 
ae Ix joy oy IQ oy 


Equating real and imaginary parts then gives the required Cauchy—Riemann equations 


du_ dy = dv __ au 
ox oy ox oy 

at the point z = z). However, z, is an arbitrarily chosen point in the region R; hence the 

Cauchy—Riemann equations hold throughout R, and we have thus proved the required 

result. 

It is tempting to think that should we choose more paths along which to let z — zy 
tend to zero, we could derive more relationships along the same lines as the Cauchy— 
Riemann equations. It turns out, however, that we merely reproduce them or expressions 
derivable from them, and it is possible to prove that satisfaction of the Cauchy—Riemann 
equations (4.19) is a necessary condition for a function f(z) = u(x, y) + ju(x, y), Z=x + Jy, 
to be analytic in a specified region. At points where f’(z) exists it may be obtained from 
either (4.20) or (4.21) as 


f@=H+j% 
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Solution 
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or 


4) =v _ du 
I(@= a j a 


If z is given in the polar form z = re’? then 
f(2) = ur, 8) + july, 9) 
and the corresponding polar forms of the Cauchy—Riemann equations are 


auld __ law 


4.22 
or roe or r 00 oe) 
At points where f’(z) exists it may be obtained from either of 
f= oi au + i@) (4.23a) 
or 
5) = ei 7m 14) 4.2 
s@=on te 1h (4.23) 


Verify that the function f(z) = z’ satisfies the Cauchy-Riemann equations, and deter- 
mine the derivative f’(z). 


Since z = x + jy, we have 
f@) = 2? = («+ jyP =? - y’) + j2xy 
so if f(z) = u(x, y) + ju(x, y) then 
u=x-y, v=2xy 


giving the partial derivatives as 


Ou Ou 
Le) LL ee, 
ox oy . 
Ov Ov 
Gy 9 
oe oy 7 


It is readily seen that the Cauchy—Riemann equations 


du_ dv du __ a 
ox oy oy Ox 
are satisfied. 


The derivative f’(z) is then given by 


Oe er 
fO= x tay 2x + j2y = 2z 


as expected. 
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Solution 
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Verify that the exponential function f(z) = e*, where @ is a constant, satisfies the 
Cauchy—Riemann equations, and show that f’(z) = ae”. 


az a(xtjy) 


ut+jv=e"=e ax eg jay — 


€ 


f@) 


so, equating real and imaginary parts, 


e* (cos ay + j sin ay) 


u =e cos ay, v =e sin ay 


The partial derivatives are 


u Dv ae 
a ae™ cos ay, Ou ae™ sin ay 
ox Ox 

1 . : 
Ws —ae™ sin ay, OF ae” cos ay 
oy oy 


confirming that the Cauchy—Riemann equations are satisfied. The derivative f’(z) is 
then given by 


f@= du + jz = ae (cos ay +jsin ay) = ae” 
Ox ox 
so that 
A ee = Get (4.24) 
dz 


As in the real variable case, we have (see Section 4.3.1) 


eF =cosz+jsinz (4.25) 
so that cosz and sinz may be expressed as 
ete 
cos z= ; 
(4.26a) 
. ee! 
sin Z F 
Using result (4.24) from Example 4.10, it is then readily shown that 
i) (sinz) = cosz 
dz 
a (cos z) = —sinz 
dz 
Similarly, we define the hyperbolic functions sinhz and coshz by 
sinhz = ae = -jsinjz 
a (4.26b) 
coshz = a = COS jz 
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from which, using (4.24), it is readily deduced that 


as (sinh z) = coshz 
dz 


D (soutiey Sint 
dz 


We note from above that e* has the following real and imaginary parts: 
Re(e’) = e* cosy 
Im(e*) = e* siny 


In real variables the exponential and circular functions are contrasted, one being mono- 
tonic, the other oscillatory. In complex variables, however, the real and imaginary parts 
of e° are (two-variable) combinations of exponential and circular functions, which 
might seem surprising for an exponential function. Similarly, the circular functions of 
a complex variable have unfamiliar properties. For example, it is easy to see that |cosz| 
and |sinz| are unbounded for complex z by using the above relationships between 
circular and hyperbolic functions of complex variables. Contrast this with |cosx| < 1 
and |sinx| < 1 for a real variable x. 

In a similar way to the method adopted in Examples 4.9 and 4.10 it can be shown 
that the derivatives of the majority of functions f(x) of a real variable x carry over to the 
complex variable case f(z) at points where f(z) is analytic. Thus, for example, 


n n-l 


d on 
z"=nz 
d 


for all z in the z plane, and 


ie 
Zz 


dz 


for all z in the z plane except for points on the non-positive real axis, where Inz is 
non-analytic. 

It can also be shown that the rules associated with derivatives of a function of a real 
variable, such as the sum, product, quotient and chain rules, carry over to the complex 
variable case. Thus, 


£1 fie) + et) = ena de) 
Z, dz dz 


Sif) 82] = fz) 8D + 1D 92) 
z dz dz 
d Bdjide 
ieee 


d Fall _ g2f@=f/@) 2) 


dz| g(z) | (gp 
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4.3.2 


Example 4.11 


Solution 


Conjugate and harmonic functions 


A pair of functions u(x, vy) and v(x, y) of the real variables x and y that satisfy the 
Cauchy—Riemann equations (4.19) are said to be conjugate functions. (Note here 
the different use of the word ‘conjugate’ to that used in complex number work, where 
z* = x — jy is the complex conjugate of z = x + jy.) Conjugate functions satisfy the 
orthogonality property in that the curves in the (x, y) plane defined by u(x, v) = constant 
and v(x, y) = constant are orthogonal curves. This follows since the gradient at any point 
on the curve u(x, vy) = constant is given by 


dy] __du 2 
dx] dy| ox 


and the gradient at any point on the curve v(x, y) = constant is given by 


dy] _ _dv |x 
dx dy| Ox 


It follows from the Cauchy—Riemann equations (4.19) that 


dy| |dy| _ _y 
dx | dx | 


so the curves are orthogonal. 
A function that satisfies the Laplace equation in two dimensions is said to be 
harmonic; that is, w(x, y) is a harmonic function if 


Ju fe Ju =0 
ax ay” 
It is readily shown (see Example 4.12) that if f(z) = u(x, y) + ju(, y) is analytic, so that 
the Cauchy—Riemann equations are satisfied, then both u and v are harmonic functions. 
Therefore wu and v are conjugate harmonic functions. Harmonic functions have applica- 
tions in such areas as stress analysis in plates, inviscid two-dimensional fluid flow and 


electrostatics. 


Given u(x, y) =x° — y’ + 2x, find the conjugate function v(x, y) such that f(z) = 
u(x, y) + jv(x, y) is an analytic function of z throughout the z plane. 


We are given u(x, y)=x° — y’ + 2x, and, since f(z) = u + jv is to be analytic, the Cauchy— 
Riemann equations must hold. Thus, from (4.19), 


Ov _ Ou _ 
oo 


Integrating this with respect to y gives 
v = 2xy + 2y + F(x) 


where F(x) is an arbitrary function of x, since the integration was performed holding 
x constant. Differentiating v partially with respect to x gives 


www.2Ofile.org 


4.3 COMPLEX DIFFERENTIATION 289 


Example 4.12 


Solution 


Ov dF 
W_ 4,408 
Ox vw a 


but this equals —du/dy by the second of the Cauchy—Riemann equations (4.19). Hence 


But since u = x* — y’ + 2x, du/dy =—2y, and comparison yields F(x) = constant. This 
constant is set equal to zero, since no conditions have been given by which it can be 
determined. Hence 


u(x, vy) + ju(x, y) = vr - y + 2x + j(2xy + 2y) 


To confirm that this is a function of z, note that f(z) is f(x + jy), and becomes just /(x) 
if we set y= 0. Therefore we set y = 0 to obtain 


f(x + j0) = f(x) = u(x, 0) + ju(x, 0) =x° + 2x 
and it follows that 
f@=27 +22 


which can be easily checked by separation into real and imaginary parts. 


Show that the real and imaginary parts u(x, v) and v(x, y) of a complex analytic function 
f(%) are harmonic. 


Since 
Sf (2) = ux, y) + ju, y) 
is analytic, the Cauchy—Riemann equations 
dv_ du du_ a 
ox oy’ ox oy 
are satisfied. Differentiating the first with respect to x gives 
dv du | Ju _ =| >) 
Ox 


ax axdy~— Oydx~—ay 
which is —0°v/dy*, by the second Cauchy—Riemann equation. Hence 


» or 0 
ox oy” ax oy 


and v is a harmonic function. 
Similarly, 


Bi O(N) 


ov dyax 7 “a oy ax 


so that 
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dv. dv 

ao ae 

ox” oy 
and u is also a harmonic function. We have assumed that both uw and v have continuous 
second-order partial derivatives, so that 


=0 


a a ee 
Oxdy — dyax’ Ooxdy  dvax 
4.3.3 Exercises 
24 Determine whether the following functions are 29 Find the orthogonal trajectories of the following 


analytic, and find the derivative where appropriate: families of curves: 


(a) ze (b) sin 4z (a) xy—xy=a (constant a) 
(c) zz* (d) cos 2z (b) e“cosy+xy=a (constant ©) 
25 Determine the constants a and b in order that 30 Find the real and imaginary parts of the functions 
w =x + ay — 2xy + j(bx’ —y + 2xy) (a) 2c 
be analytic. For these values of a and b find the (b) sin2z 


derivative of w, and express both w and dw/dz as 


functions of z=x + jy. Verify that they are analytic and find their 


derivatives. 


26 ‘Finda function v(x, y) such that, given uw =2x(1—y), 
f(z) =u + jv is analytic in z. 31. Give a definition of the inverse sine function 


sin'z for complex z. Find the real and imaginary 


Zi 


Show that @(x, vy) =e*(x cos y—y sin y) 1s a harmonic 
function, and find the conjugate harmonic function 
w(x, y). Write o(x, vy) + j wx, y) as a function of 


parts of sin”! z. (Hint: put z = sin w, split into 
real and imaginary parts, and with w =u + jv 
and z =x + jy solve for u and v in terms of x 


and y.) Is sin'z analytic? If so, what is its 
derivative? 


z=x+Jy only. 


28 — Show that u(x, y) = sinx coshy is harmonic. Find 
the harmonic conjugate v(x, y) and express w=u+jv Sy 
as a function of z = x + jy. 


Establish that if z = x + jy, 
|sinhy| S |sinz| S coshy. 


4.3.4 Mappings revisited 


In Section 4.2 we examined mappings from the z plane to the w plane, where in the 
main the relationship between w and z, w = f(z) was linear or bilinear. There is an 
important property of mappings, hinted at in Example 4.8 when considering the map- 
ping w=z°*. A mapping w = f(z) that preserves angles is called conformal. Under such 
a mapping, the angle between two intersecting curves in the z plane is the same as the 
angle between the corresponding intersecting curves in the w plane. The sense of the 
angle is also preserved. That is, if @ is the angle between curves | and 2 taken in the anti- 
clockwise sense in the z plane then @ is also the angle between the image of curve | 
and the image of curve 2 in the w plane, and it too is taken in the anticlockwise sense. 
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Figure 4.15 
Conformal mappings. 


Example 4.13 


Solution 
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w=f(z) 
Y (conformal) u 
curve 2 flearve 
6 
curve | f(curve 1) 
f(Zo) 


z plane w plane 


Figure 4.15 should make the idea of a conformal mapping clearer. If f(z) is analytic 
then w = f(z) defines a conformal mapping except at points where the derivative f’(z) 
is zero. 

Clearly the linear mappings 


w=az+B (a#0) 


are conformal everywhere, since dw/dz = @ and is not zero for any point in the z plane. 
Bilinear mappings given by (4.12) are not so straightforward to check. However, as we 
saw in Section 4.2.4, (4.12) can be rearranged as 


w=a+—4— (a, #0) 


azt+ B 
Thus 
dw HO 
dz (az+B) 


which again is never zero for any point in the z plane. In fact, the only mapping we have 
considered so far that has a point at which it is not conformal everywhere is w = z’ 
(cf. Example 4.8), which is not conformal at z = 0. 


Determine the points at which the mapping w =z + 1/z is not conformal and demon- 
strate this by considering the image in the w plane of the real axis in the z plane. 


Taking z=x+jy and w=u + jv, we have 
x—jy 
x+y 


Ww=utjv=xtjyt 


which, on equating real and imaginary parts, gives 


u=xt+ ——~— 
x+y 

~ y 

ar amma 2 
x+y 
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Figure 4.16 Image 
ofz=1+ eof 
Example 4.13. 


The real axis, y = 0, in the z plane corresponds to v = 0, the real axis in the w plane. 
Note, however, that the fixed point of the mapping is given by 


Z=Z+ 1 
Zz 

or z = ce. From the Cauchy—Riemann equations it is readily shown that w is analytic 
everywhere except at z= 0. Also, dw/dz = 0 when 

1-4=0, thatis 2=+1 

Zz 

which are both on the real axis. Thus the mapping fails to be conformal at z = 0 and 
z=. The image of z= | is w = 2, and the image of z = —1 is w = —2. Consideration 
of the image of the real axis is therefore perfectly adequate, since this is a curve passing 
through each point where w =z + 1/z fails to be conformal. It would be satisfying if we 
could analyse this mapping in the same manner as we did with w = z’ in Example 4.8. 
Unfortunately, we cannot do this, because the algebra gets unwieldy (and, indeed, our 
knowledge of algebraic curves is also too scanty). Instead, let us look at the image of 
the point z = 1 + €, where € is a small real number. € > 0 corresponds to the point Q 
just to the right of z = | on the real axis in the z plane, and the point P just to the 
left of z = 1 corresponds to € < 0 (Figure 4.16). 


z plane 


Ifz=1+ethen 


et es ate 
l+é 


=lt+e+(l+e)'! 
=lt+et+l—-ete-e +t... 
=2+e 


if | €| is much smaller than 1 (we shall discuss the validity of the power series expansion 
in Section 4.4). Whether € is positive or negative, the point w = 2 + €’ is to the right of 
w = 2 in the w plane as indicated by the point R in Figure 4.16. Therefore, as € > 0, a 
curve (the real axis) that passes through z = | in the z plane making an angle 0= 7 
corresponds to a curve (again the real axis) that approaches w = 2 in the w plane along 
the real axis from the right making an angle @= 0. Non-conformality has thus been 
confirmed. The treatment of z = —1 follows in an identical fashion, so the details 
are omitted. Note that when y= 0 (v= 0), u=x+ 1/x so, as the real axis in the z plane 
is traversed from x = —co to x = 0, the real axis in the w plane is traversed from 
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Figure 4.17 Image 
in w plane of the real 
axis in the z plane for 
Example 4.13. 


Example 4.14 


Solution 
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u =—ce to —2 and back to u = —°° again (when x = —1, u reaches —2). As the real 
axis in the z plane is traversed from x = 0 through x = | to x = +e, so the real axis in 
the w plane is traversed from u = +°° to u = +2 (x = 1) back to u = © again. Hence the 
points on the real axis in the w plane in the range —-2 < u < 2 do not correspond to real 
values of z. Solving u =x + 1/x for x gives 


<= Slut (Ww — 4)] 


which makes this point obvious. Figure 4.17 shows the image in the w plane of the real 
axis in the z plane. This mapping is very rich in interesting properties, but we shall not 
pursue it further here. Aeronautical engineers may well meet it again if they study the 
flow around an aerofoil in two dimensions, for this mapping takes circles centred at the 
origin in the z plane onto meniscus (lens-shaped) regions in the w plane, and only a 
slight alteration is required before these images become aerofoil-shaped. 


Examine the mapping 
wH=e 


by (a) finding the images in the w plane of the lines x = constant and y = constant in 
the z plane, and (b) finding the image in the w plane of the left half-plane (x < 0) in the 
z plane. 


Taking z=x+jy and w=u+ Jv, for w=e° we have 
u=e'cosy 
v=e'siny 

Squaring and adding these two equations, we obtain 
wt+v=e* 


On the other hand, dividing the two equations gives 
v= tany 
u 


We can now tackle the questions. 


(a) Since uv? +v* = e™, putting x = constant shows that the lines parallel to the imagin- 
ary axis in the z plane correspond to circles centred at the origin in the w plane. 
The equation 


Vv 
- =tany 
u 


shows that the lines parallel to the real axis in the z plane correspond to straight 
lines through the origin in the w plane (v = utana@ if y = a, a constant). 
Figure 4.18 shows the general picture. 
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Figure 4.18 Mapping 
of lines under w = e’. 


y=Im (z) 


z plane w plane 
(b) Since uw’ + v? = e*, if x = 0 then uv’ + v’ = 1, so the imaginary axis in the z plane 
corresponds to the unit circle in the w plane. If x < 0 then e* < 1, and as x > —~, 
e** — 0, so the left half of the z plane corresponds to the interior of the unit circle 
in the w plane, as illustrated in Figure 4.19. 
Figure 4.19 Mapping y : 
w=e 
of sig a under ess ae oe 
w=e. 
> > 
O x -1 1 u 
z plane w plane 


SS 


34 


35) 


4.3.5 Exercises 


Determine the points at which the following 
mappings are not conformal: 


(a) w=2-1 (b) w= 223 — 212° +727 +6 


(c) w= 8244 
2z 


Follow Example 4.13 for the mapping w =z — 1/z. 
Again determine the points at which the mapping is 
not conformal, but this time demonstrate this by 
looking at the image of the imaginary axis. 


Find the region of the w plane corresponding to 
the following regions of the z plane under the 
exponential mapping w = e*: 

(a) OX x< 00 (b) OSx¥<1,0SyS1 


(c) in<y<2,0<x<0 


36 


Sy 


Consider the mapping w = sinz. Determine the 
points at which the mapping is not conformal. 

By finding the images in the w plane of the 

lines x = constant and y = constant in the z plane 
(z =x +jy), draw the mapping along similar lines to 
Figures 4.14 and 4.18. 


Show that the transformation 
2 
a 


4 


where z = x + jy and €= Re’® maps a circle, with 
centre at the origin and radius a, in the € plane, onto 
a straight line segment in the z plane. What is the 
length of the line? What happens if the circle in the 
¢ plane is centred at the origin but is of radius b, 
where b 4a? 


z=C+ 
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re Complex series 


4.4.1 


In Modern Engineering Mathematics we saw that there were distinct advantages in being 
able to express a function f(x), such as the exponential, trigonometric and logarithmic 
functions, of a real variable x in terms of its power series expansion 


f= By AyX" =AgtAX+ AX? +...+4,x" +... (4.27) 


n=0 


Power series are also very important in dealing with complex functions. In fact, any real 
function f(x) which has a power series of the form in (4.27) has a corresponding com- 
plex function f(z) having the same power series expansion, that is 


{®™= >) 4,2" =Aytayztanzt+...+4,2"+... (4.28) 
n=0 


This property enables us to extend real functions to the complex case, so that methods 
based on power series expansions have a key role to play in formulating the theory of 
complex functions. In this section we shall consider some of the properties of the power 
series expansion of a complex function by drawing, wherever possible, an analogy with 
the power series expansion of the corresponding real function. 


Power series 


A series having the form 


>> a, (Z — 2)" = Ay + a(Z— 2) + a,(z—-Z)Yt+...+4,(z—Z)' +... (4.29) 


n=0 


in which the coefficients a, are real or complex and z, is a fixed point in the complex 
z plane is called a power series about z, or a power series centred on z). Where z) = 0, 
the series (4.29) reduces to the series (4.28), which is a power series centred at the 
origin. In fact, on making the change of variable z’ = z — zp, (4.29) takes the form (4.28), 
so there is no loss of generality in considering the latter below. 

Tests for the convergence or divergence of complex power series are similar to those 
used for power series of a real variable. However, in complex series it is essential that 
the modulus | a,,| be used. For example, the geometric series 


— n 
2 


n=0 


has a sum to N terms 


and converges, if|z| < 1, to the limit 1/(1 —z) as N > o. If |z| = 1, the series diverges. 
These results appear to be identical with the requirement that |x| < 1 to ensure con- 
vergence of the real power series 
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Figure 4.20 
Region of 
convergence 
of Y 2”. 


n=0 


However, in the complex case the geometrical interpretation is different in that the 
condition |z| < 1 implies that z lies inside the circle centred at the origin and radius 1 
in the z plane. Thus the series ¥,_)z” converges if z lies inside this circle and diverges 
if z lies on or outside it. The situation is illustrated in Figure 4.20. 


vA 
iii |2\=1 
¢ 7 


¢ » 
¢ x 
a \ 
' ° 
; Series converges Series 
T 
‘ 


‘ 
1 
O | diverges ‘x 


- 
s 


The existence of such a circle leads to an important concept in that in general there 
exists a circle centred at the origin and of radius R such that the series 


= . converges if |z| <R 
divergesif |z| >R 


The radius R is called the radius of convergence of the power series; what happens 
when |z| = & is normally investigated as a special case. 

We have introduced the radius of convergence based on a circle centred at the 
origin, while the concept obviously does not depend on the location of the centre of 
the circle. If the series is centred on z, as in (4.29) then the convergence circle would 
be centred on Zp). Indeed it could even be centred at infinity, when the power series 
becomes 


which we shall consider further in Section 4.4.5. 

In order to determine the radius of convergence R for a given series, various tests for 
convergence, such as those introduced in Modern Engineering Mathematics for real 
series, may be applied. In particular, using d’ Alembert’s ratio test, it can be shown that 
the radius of convergence R of the complex series ¥,-).a,z” is given by 
an 


R= lim 


n—-co 


(4.30) 


Qn+1 


provided that the limit exists. Then the series is convergent within the disc |z| < R. 
In general, of course, the limit may not exist, and in such cases an alternative method 
must be used. 
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Solution 
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Find the power series, in the form indicated, representing the function 1/(z — 3) in the 
following three regions: 


(a) |z| <3; 


n=0 
(c) |z| > 3; a 


and sketch these regions on an Argand diagram. 


We know that the binomial series expansion 


(1+2)'=14n2+ Mee D 24... + We Dm=2 — PN 
i: | ae 


is valid for |z| < 1. To solve the problem, we exploit this result by expanding the 
function 1/(z — 3) in three different ways: 


1 
1 ~3 - n 
e. <= == = —h(1-42)' = 1 t4z4(izy’ +... 4 (42)"+...] 
3 
for |+z| < 1, that is |z| < 3, giving the power series 
1 
pogo ne 21 <3) 
1 1 -1 
b) —e = = [(2-2)-1 
(b) 3 eo 1 [(z-2)-1] 
=-[l+(z-2)+(z-27 +...] (lz-21<D 
giving the power series 
a ey (jz-2| <1) 
Z=3 
1 _i/z_ _1 3,(3/ 
(©) ae 


giving the power series 


(1z| > 3) 


The three regions are sketched in Figure 4.21. Note that none of the regions includes 
the point z= 3, which is termed a singularity of the function, a concept we shall discuss 
in Section 4.5.1. 
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Figure 4.21 Regions 
of convergence for the 
series in Example 4.15. 


Example 4.16 


Solution 


z plane 


In Example 4.15 the whole of the circle |z| = 3 was excluded from the three regions 
where the power series converge. In fact, it is possible to include any selected point in 
the z plane as a centre of the circle in which to define a power series that converges 
to 1/(z — 3) everywhere inside the circle, with the exception of the point z = 3. For 
example, the point z = 4j would lead to the expansion of 


I | I 
z—4j+4j-3 4j)-3z-4),, 
4j - 3 


in a binomial series in powers of (z — 4j)/(4j — 3), which converges to 1/(z — 3) inside 
the circle 


la=4j/=]4j=31= 70642) =5 


We should not expect the point z = 3 to be included in any of the circles, since the 
function 1/(z — 3) is infinite there and hence not defined. 


Prove that both the power series );a,z” and the corresponding series of derivatives 
YF) na,z"' have the same radius of convergence. 


Let R be the radius of convergence of the power series );, a,,z”. Since lim,,_,.. (a,z3) = 0 
(otherwise the series has no chance of convergence), if |Z)| < R for some complex number 
Z, then it is always possible to choose 


|a,] < |Z” 


for n > N, with N a fixed integer. We now use d’Alembert’s ratio test, namely 


: . {a — 

if lim|4#) <1 then Ya," converges 
n—-oo a), = 

, . QAn+1 — n : 

if lim|“"| > 1 then Yi a2 diverges 
noo a, 


n=0 
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The differentiated series },na,z"' satisfies 


co co ee n-1 
by |na,z” '| < yn a,| |z|"" < D3 Pie 
n=1 n=1 


n=1 | Z|” 


which, by the ratio test, converges if 0 < |z)| < R, since |z| < |z)| and |z)| can be as 
close to R as we choose. If, however, |z| > R then lim,,.,..(a,z") # 0 and thus 
lim,,,.. (na,z""') # 0 too. Hence R is also the radius of convergence of the differentiated 
series )_,na,z"". 


The result obtained in Example 4.16 is important, since if the complex function 


fe=>, 4,2" 


n=0 


converges in |z| < R then the derivative 


co 


f@= > naz" 
n=1 
also converges in |z| < R. We can go on differentiating f(z) through its power series 
and be sure that the differentiated function and the differentiated power series are equal 
inside the circle of convergence. 


4.4.2 Exercises 


Find the power series representation for the 39 Find the power series representation of the function 
function 1/(z — j) in the regions 
f= 

(a) |z|<1 ee 
(b) |z|> 1 in the disc |z| < 1. Use Example 4.16 to deduce the 
(c) |z—1-j|< 2 power series for 
Deduce that the radius of convergence of the 1 1 

: f . aoe (a) ————— (b) ———. 
power series representation of this function is es 1? ies 1) 
|Z) - jl, where z = zy is the centre of the circle of 
convergence (Z) # j). valid in this same disc. 


4.4.3 Taylor series 


In Modern Engineering Mathematics we introduced the Taylor series expansion 


2 co n 
f+ a)=fla) + a fa) + i fa@)+...5F a fa) (4.31) 
n=0 
of a function f(x) of a real variable x about x = a and valid within the interval of con- 
vergence of the power series. For the engineer the ability to express a function in such 
a power series expansion is seen to be particularly useful in the development of numer- 
ical methods and the assessment of errors. The ability to express a complex function as 
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y 


xy 


z plane 


Figure 4.22 Region 
of convergence of the 
Taylor series. 


Example 4.17 


Solution 


a Taylor series is also important to engineers in many fields of applications, such as 
control and communications theory. The form of the Taylor series in the complex case 
is identical with that of (4.31). 

If f(z) is a complex function analytic inside and on a simple closed curve C (usually 
a circle) in the z plane then it follows from Example 4.16 that the higher derivatives of 
f(2) also exist inside C. If z) and z, + / are two fixed points inside C then 


Ney +H) = fled) + hfe) + EPG) +... + Hp Mee.. 
: Nn. 


where f(z,) is the kth derivative of f(z) evaluated at z = zy. Normally, z = z) + h is 
introduced so that 4 = z — zp, and the series expansion then becomes 


fle) = flex) + © - a) FE) + FSH p1G,) +... 


SE) ey. eal egy (4.32) 


n=0 


The power series expansion (4.32) is called the Taylor series expansion of the com- 
plex function f(z) about z). The region of convergence of this series is |z — z)| < R, 
a disc centred on z = z, and of radius R, the radius of convergence. Figure 4.22 
illustrates the region of convergence. When z, = 0, as in real variables, the series expan- 
sion about the origin is often called a Maclaurin series expansion. 

Since the proof of the Taylor series expansion does not add to our understanding 
of how to apply the result to the solution of engineering problems, we omit it at this 
stage. 


Determine the Taylor series expansion of the function 


1 


$= ay 


about the point z = j: 


(a) directly up to the term (z — j)’, 


(b) using the binomial expansion. 


Determine the radius of convergence. 


(a) The disadvantage with functions other than the most straightforward is that 
obtaining their derivatives is prohibitively complicated in terms of algebra. 
It is easier in this particular case to resolve the given function into partial 
fractions as 


opt eg bf tt 
A) =e op rears ) 
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The right-hand side is now far easier to differentiate repeatedly. Proceeding to 
determine f(j), we have 


fey=t(—- 4), — sothat f(i)=1 

2j\z-2j) 2 

f%) = a oe + . so that f'(j)=0 
2j| «@-2jy 2 |’ 

(7 ea 3 ,  sothat fj) =-2 
1[ 6 6 

Gla @— oy +), so that fj) =0 

I= 5 a “) so that fj) =24 


leading from (4.32) to the Taylor series expansion 


reed: rr 
z(z — 2j) 
=1-(z-jyt+(z-jy't+... 


2 2, 24 “4 
ary ee) eed) eee 


(b) To use the binomial expansion, we first express z(z — 2j) as (z —j +j)\(z-J —J), 
which, being the difference of two squares ((z — j)’ — j°), leads to 


fe=— t+ @-5yy 


xz-2j) (-jy+1 
Use of the binomial expansion then gives 
i@el-Gaiy +e) 21) ts 


valid for |z —j| < 1, so the radius of convergence is 1. 


The points where f(z) is infinite (its singularities) are precisely at distance | away 
from z = j, so this value for the radius of convergence comes as no surprise. 


Example 4.18 — Suggest a function to represent the power series 


z Zs z" 
es i Gee rr alee ee 
2! 3! n! 


and determine its radius of convergence. 


Solution Set 


2 3 co 
= ae =VvZ2 
f=l4+z+ Te ae La 
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Assuming we can differentiate the series for f(z) term by term, we obtain 


co n-1 co n-1 


f'@= esas Lao 


Hence f(z) is its own derivative. Since e* is its own derivative in real variables, and is 
the only such function, it seems sensible to propose that 


f(z) = s 5 =e (4.33) 


the complex exponential function. Indeed the complex exponential e* is defined by 
the power series (4.33). According to d’Alembert’s ratio test the series ya, is 
convergent if |a,,,/a,| > L < 1asn—, where L is a real constant. If a, = z"/n! then 
|a,,,/a,| =|z|/(n + 1) which is less than unity for sufficiently large n, no matter how 
big |z| is. Hence );,2”/n! is convergent for all z and so has an infinite radius of con- 
vergence. Note that this is confirmed from (4.30). Such functions are called entire. 


In the same way as we define the exponential function e* by the power series expan- 
sion (4.31), we can define the circular functions sinz and cosz by the power series 


expansions 
z z z 
snz=z—->+>-=>+..., 
3! 5! O77! 
2 4 6 
cosz=1-242-2 
2! 4! 6! 


both of which are valid for all z. Using these power series definitions, we can readily 


prove the result (4.25), namely 


e¥ = cosz+jsinz 


4.4.4 Exercises 


4O Find the first four non-zero terms of the Taylor 
series expansions of the following functions about 
the points indicated, and determine the radius of 
convergence in each case: 


~=1) ()— 


Tas a2— 4j) 


(z= 2) 
1 ? 

() = @=1+)) 
Zz 


41 Find the Maclaurin series expansion of the function 


a 
l+z4+27 


SZ) = 


up to and including the term in z’*. 


Without explicitly finding each Taylor series 
expansion, find the radius of convergence of 
the function 


(Nes 
about the three points z= 0, z=1+jandz=2 +4 2j. 
Why is there no Taylor series expansion of this 
function about z = j? 


Determine a Maclaurin series expansion 
of f(z) = tanz. What is its radius of 
convergence? 
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4.4.5 Laurent series 


Figure 4.23 
The Riemann sphere. 


Figure 4.24 Region of 
validity of the Laurent 
series. 


Let us now examine more closely the solution of Example 4.15(c), where the power 
series obtained was 

eel tat dy. 

z-3 2 2 3 
valid for |z| > 3. In the context of the definition, this is a power series about ‘z = ©’, 
the ‘point at infinity’. Some readers, quite justifiably, may not be convinced that there 
is a single unique point at infinity. Figure 4.23 shows what is termed the Riemann 
sphere. A sphere lies on the complex z plane, with the contact point at the origin O. Let 
O’ be the top of the sphere, at the diametrically opposite point to O. Now, for any 
arbitrarily chosen point P in the z plane, by joining O’ and P we determine a unique 
point P’ where the line O’P intersects the sphere. There is thus exactly one point P’ on 
the sphere corresponding to each P in the z plane. The point O’ itself is the only point 
on the sphere that does not have a corresponding point on the (finite) z plane; we there- 
fore say it corresponds to the point at infinity on the z plane. 


oO’ 


z plane 


Returning to consider power series, we know that, inside the radius of convergence, 
a given function and its Taylor series expansion are identically equal. Points at which 
a function fails to be analytic are called singularities, which we shall discuss in 
Section 4.5.1. No Taylor series expansion is possible about a singularity. Indeed, a 
Taylor series expansion about a point z) at which a function is analytic is only valid 
within a circle, centre z), up to the nearest singularity. Thus all singularities must be 
excluded in any Taylor series consideration. The Laurent series representation includes 
(or at least takes note of) the behaviour of the function in the vicinity of a singularity. 

If f(z) is a complex function analytic on concentric circles C, and C, of radii r, and 
ry (with r, < r,), centred at z), and also analytic throughout the region between the 
circles (that is, an annular region), then for each point z within the annulus (Figure 4.24) 
(2) may be represented by the Laurent series 


Yh 
CQ 
e 
= 
oO x 
z plane 


www.2Ofile.org 


304 FUNCTIONS OF A COMPLEX VARIABLE 


SZ) ae y Ce a Zo) 


n=—co 


C_, ais C_y41 tee C_\ (4.34) 


(z-Z)" (z- 29)" Z- Zo 


tp C(B= 2) Poo cth@l@=Ayl tbeoc 


Sac par 


where in general the coefficients c, are complex. The annular shape of the region is 
necessary in order to exclude the point z = z), which may be a singularity of f(z), from 
consideration. If f(z) is analytic at z = z, then c, = 0 for n = —1, —2,..., —ce, and the 
Laurent series reduces to the Taylor series. 

The Laurent series (4.34) for f(z) may be written as 


-1 co 
2) = ¥, enlz = 20)" + ¥ en(Z = 20)" 
n=—00 n=0 
and the first sum on the right-hand side, the ‘non-Taylor’ part, is called the principal 
part of the Laurent series. 

Of course, we can seldom actually sum a series to infinity. There is therefore often more 
than theoretical interest in the so-called ‘remainder terms’, these being the difference 
between the first n terms of a power series and the exact value of the function. For 
both Taylor and Laurent series these remainder terms are expressed, as in the case of 
real variables, in terms of the (” + 1)th derivative of the function itself. For Laurent series 
in complex variables these derivatives can be expressed in terms of contour integrals 
(Section 4.6), which may be amenable to simple computation. Many of the details are 
outside the scope of this book, but there is some introductory material in Section 4.6. 


Example 4.19 For f(z) = 1/z’(z + 1) find the Laurent series expansion about (a) z = 0 and (b) z=—1. 
Determine the region of validity in each case. 


Solution As with Example 4.15, problems such as this are tackled by making use of the binomial 
series expansion 


(1+2)"=14nz+ 8448 n-1 latin n-rt+l aa 


provided that |z| < 1. 


(a) In this case z) = 0, so we need a series in powers of z. Thus 


tw br ezy 

z(1+z) 2z 
=1a-z4+27-24+2-...) O<lz<) 
Zz 


Thus the required Laurent series expansion is 
Sint oe Bere eee 
z(z+1) 2 2 
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Solution 


(b) 
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valid for 0 < |z| < 1. The value z = 0 must be excluded because of the first two 
terms of the series. The region 0 < |z| < 1 is an example of a punctured disc, a 
common occurrence in this branch of mathematics. 
In this case z) = —1, so we need a series in powers of (z + 1). Thus 
| eevee eee 
Z(z+1) (z+) 
__ 1 
(z+ 1) 
1 2 
= 14+2(2+1)+3(2+1)+... 
Gap [ @+1)+3G+1) ] 


=o $24+3(c+1) +4241) +... 


(¢+1-1)? 


e+" 


valid for 0 < |z + 1] < 1. Note that in a meniscus-shaped region (that is, the 
region of overlap between the two circular regions |z| < 1 and|z+ 1] < 1) both 
Laurent series are simultaneously valid. This is quite typical, and not a cause for 
concern. 


Determine the Laurent series expansions of 


as oe WEE 3) 
valid for 
(a) 1<|z|<3 
(b) [z|>3 
(c) O<|z+1[<2 
(d) |z|<1 
(a) Resolving into partial functions, 


fey = (5) Lz) 


Since |z| > 1 and |z| < 3, we express this as 


f@)= CEST 7 Ges 


2z 
eee ae ve AM te ae 4) 
1 1 1 1 2 3 
= Se a ae 


274 22° 22? 2 
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() fe)= laa) lees) 


Since |z| > 3, we express this as 


igs s(a-47)- Cre) 


al z 
=2(1-4+5-44...)-2(1-24+3-8 } 
27 aA Zz 2z Z 2° Z 

1 4 13 40 
a a es 

ZZ Zz Zz 


(c) We can proceed as in Example 4.18. Alternatively, we can take z+ 1 = uw; then 
0 < |u| <2 and 


1 1 
u)= = 
flu) u(ut+2) 2u(1+4u) 
1 2 3 
=o (lope oe ) 
giving 
= 1 2 
fiz) ———— Pees 


_ 1 
ie) 1G)= a 2(z +3) 


Since |z| < 1, we express this as 


a on 
eee 6(1 + 42) 


= H(1 +z) -h1+4z)" 


=M1-ztz°-z7+.. .)-3 Wy tee t-te, .) 


f@) = 


a 13,2 _ 40,3 
=3 ees age ge Ps 


Example 4.21 Determine the Laurent series expansion of the function f(z) = z’e'” about 
(a) z=0 
(b) z=a,a finite, non-zero complex number 


(c) z= 


Solution (a) From (4.33), 


2 
oe) ae (0 <|z| < ~) 
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Substituting 1/z for z, we obtain 


ate tete +... (0 < |z| =) 
z lz 
so that 
3 qliz 3 2 ! : : 
Peart r7t+—+—4+—+4+ +... (0< [2] = &%) 


2! 3! 4lz Siz 


This series has infinitely many terms in its principal part, but stops at z° (it is 
written back to front). Series with never-ending principal parts are a problem, and 
fortunately are uncommon in engineering. Note also that the series is valid in an 
infinite punctured disc. 


The value of f(a) must be ae’, which is not infinite since a # 0. Therefore f(z) 
has a Taylor series expansion 


fe) = Na+ af a)+ BEB pray... 
about z = a. We have 


f°) = Ff (z? e!”) = 377 el _ zel? 
Z 


f°) = < (2 el” == zel”) = 6ze!” = 4el? a 1 el? 
Zz 


giving the series as 


2 el@ = a el’ 4 (z = ay3a el4 -~a e!“) 


‘ iG : a) (6a eae + a e'") oe 


which is valid in the region |z — a| < R, where R is the distance between the 
origin, where f(z) is not defined, and the point a; hence R = |a|. Thus the region 
of validity for this Taylor series is the disc |z — a] < |a|. 


To expand about z = ©, let w = 1/z, so that 
_ 1 Ww 
fl) =e 
w 
Expanding about w = 0 then gives 
a ] 
p(t) =Litws 4 eee...) 
we ow 2! 3! 
=—t+—t+ +++... (0<|wl| <9) 
4! 


Note that this time there are only three terms in the principal part of f(z)(=f(/w)). 
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4.4.6 Exercises 


44 Determine the Laurent series expansion of (a) z=0 (b) z= 
1 c) z=a, a finite non-zero complex number 
flz) = —— " 
2(z- 1) (For (c), do not calculate the coefficients explicitly.) 

about (a) z = 0 and (b) z= 1, and specify the region 

of validity for each. 46 Expand 
45 Determine the Laurent series expansion of the f@= 2. 

function (z-1)(2-z) 


1 in a Laurent series expansion valid for 
z)=z° sin= 

f@) 
Z (a) |z|<1 (b) 1<|z|<2 (c) |z| >2 


about the points (d) |z-1]>1 (e) 0<|z-2/<1 


Singularities, zeros and residues 


4.5.1 Singularities and zeros 


As indicated in Section 4.4.5 a singularity of a complex function f(z) is a point of 
the z plane where f(z) ceases to be analytic. Normally, this means f(z) is infinite at such 
a point, but it can also mean that there is a choice of values, and it is not possible to 
pick a particular one. In this chapter we shall be mainly concerned with singularities 
at which f(z) has an infinite value. A zero of f(z) is a point in the z plane at which 
f@ = 0. 

Singularities can be classified in terms of the Laurent series expansion of f(z) about 
the point in question. If f(z) has a Taylor series expansion, that is a Laurent series 
expansion with zero principal part, about the point z = zp, then z, is a regular point of 
f(z). If f(2) has a Laurent series expansion with only a finite number of terms in its 
principal part, for example 


a_m a_\ 


2 ee 
(z- 2)” (Z - 20) 


f®™= +ata(z—2)+...+a,(2—-2)"+... 

then f(z) has a singularity at z = z) called a pole. If there are m terms in the principal 
part, as in this example, then the pole is said to be of order m. Another way of defining 
this is to say that z) is a pole of order m if 


lim (2 — 29)" f(@) = 4 (4.35) 


where a_,, is finite and non-zero. If the principal part of the Laurent series for f(z) at 
z= Zz, has infinitely many terms, which means that the above limit does not exist for any 
m, then z = zy is called an essential singularity of f(z). (Note that in Example 4.20 the 
expansions given as representations of the function f(z) = 1/[(z + 1)(z + 3)] in parts (a) 
and (b) are not valid at z = 0. Hence, despite appearances, they do not represent a 
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function which possesses an essential singularity at z = 0. In this case f(z) is regular at 
z= 0 with a value +.) 


If f(z) appears to be singular at z= Zz), but it turns out to be possible to define a Taylor 


series expansion there, then z = z, is called a removable singularity. The following 
examples illustrate these cases. 


(d) 


f(z) =z" has a pole of order one, called a simple pole, at z = 0. 
f(z) = (z— 1)°* has a pole of order three at z= 1. 

f(z) =e" has an essential singularity at z =}. 

The function 


z-l1 


(2422-3) 
has a zero at z= 1, a simple pole at z = —2 and a pole of order two at z = 3. 


The function 
_ sinz 
S(z) = ss 


is not defined at z = 0, and appears to be singular there. However, defining 


eee ae z)lz (z#0) 
1 (z = 0) 


gives a function having a Taylor series expansion 
sincez=1l-=—+=—-... 


that is regular at z = 0. Therefore the (apparent) singularity at z = 0 has been 
removed, and thus f(z) = (sinz)/z has a removable singularity at z = 0. 


Functions whose only singularities are poles are called meromorphic and, by and 
large, in engineering applications of complex variables most functions are meromorphic. 
To help familiarize the reader with these definitions, the following example should 
prove instructive. 


Find the singularities and zeros of the following complex functions: 


(a) 


(c) 


(a) 


1 -1 

4 wh + . (b) 4 = . . 
z-z(1+j)t+j z-z(1+j)+j 

sin(z - 1) (d 1 
z-2(1+j) +) [2-71 +j) +i] 
For 

1 
[@O=5 


z-27(1+j)tj 
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(b) 


(c) 


the numerator is never zero, and the denominator is only infinite when z is 
infinite. Thus f(z) has no zeros in the finite z plane. The denominator is zero 
when 


Z-—7(1+j)+j=0 
which factorizes to give 

(2 - 1@-j) =0 
leading to 

2=lorj 
so that the singularities are at 

z=41,-1,(1+j)A/2, (-1-j)/2 (4.36) 
all of which are simple poles since none of the roots are repeated. 
The function 

f@)= =f 

z-z(1+j)+j 
is similar to f(z) in (a), except that it has the additional term z — 1 in the numer- 
ator. Therefore, at first glance, it seems that the singularities are as in (4.36). 
However, a closer look indicates that f(z) can be rewritten as 
z-l1 

(z-1)(2+1)[z+ 40 +) 2-301 +3)1 


and the factor z— | cancels, rendering z= 1 a removable singularity, and reducing 


fz) = 


f@ to 


1 
(2+ [z+ G0 +jlz- 30 +3)] 


which has no (finite) zeros and z=—1, \}(1 + j) and \}(-1 —j) as simple poles. 


fz) = 


In the case of 
= sin(z - 1) 
> 
z-z(1+j)t+j 
the function may be rewritten as 
4O= sin(z - 1) L : 
z-1 (412+ G0 +)IE- BU +p] 


Now 


sin-1) _, | as zl 


z-1 


so once again z = | is a removable singularity. Also, as in (b), z=—l, \3 (1 +)) 
and | ; (—1 —j) are simple poles and the only singularities. However, 


sin(z — 1) =0 
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has the general solution z = 1 + Nn (V=0, +1, +2,...). Thus, apart from NV = 0, 
all of these are zeros of f(z). 


(d) For 


1 
[24-271 +j) +31 


f@) = 
factorizing as in (b), we have 
ee 
(z- 1) (zt 1) [z+ GU tp) e- Ga tr 


so—1, +1, \}(1 +j) and \}(-1 — j) are still singularities, but this time they are 
triply repeated. Hence they are all poles of order three. There are no zeros. 


f@= 


4.5.2 Exercises 


Determine the location of, and classify, the 48 Expand each of the following functions in a Laurent 
singularities and zeros of the following functions. series about z = 0, and give the type of singularity 
Specify also any zeros that may exist. (if any) in each case: 
1 = cosz 
OF Cea Os 2 
Zz (z +j) (z-Jj) z-1 


2 


. e 
(d) cothz (ce) (f) ef) (b) = 
2+n 2 


(c) z'coshz! 


z-l1 ztj 

(2) 5 2) rere Pama 
z +l (z +2) (z- 3) (d) tan'(z? + 2z + 2) 

(i) a) 49 Show that if f(z) is the ratio of two polynomials 
2(z2 - 47 +5) then it cannot have an essential singularity. 


4.5.3 Residues 


If a complex function f(z) has a pole at the point z = z, then the coefficient a_, of the 
term 1/(z — zp) in the Laurent series expansion of f(z) about z = z, is called the residue 
of f(z) at the point z = z). The importance of residues will become apparent when 
we discuss integration in Section 4.6. Here we shall concentrate on efficient ways 
of calculating them, usually without finding the Laurent series expansion explicitly. 
However, experience and judgement are sometimes the only help in finding the easiest 
way of calculating residues. First let us consider the case when f(z) has a simple pole 
at z = Z). This implies, from the definition of a simple pole, that 


a_ 


fa= 


+ ay)ta(z—2Z)+... 
Z-Zo 


in an appropriate annulus S < |z— z,| < R. Multiplying by z — z, gives 


(Z — 2) f(z) = a_, + a(z—2Z)+... 
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which is a Taylor series expansion of (z — z)) f(z). If we let z approach zp, we then obtain 


the result 
residue ata _ lim [(z — z,) (2) = a, (4.37) 
simple pole z) 72» ‘ 


Hence evaluating this limit gives a way of calculating the residue at a simple pole. 


Example 4.23 Determine the residues of 


2z 


fe)= => — 
(2 +1)(2z-1) 


at each of its poles in the finite z plane. 


Solution Factorizing the denominator, we have 


_ 2z 
(z-j)(2+j)(2z- 1) 
so that f(z) has simple poles at z = j, —j and ; . Using (4.37) then gives 


f(2) 


residue _ jy. ezrin 2z 
atz=j 23 (z -j)(z+j)(2z- 1) 
ee eee a2) 
2j(2j - 1) 5 
residue _ lin a 22 
atz=-J (z -j)(z+j)(2z - 1) 
eee eee eee 
—2j(-2j- 1) 5 
ie = lim (z= 4) 2 
alz= 7 24 (z-j)(Z +))(@- 9) 
l 
Gover * 


Note in this last case the importance of expressing 2z — | as 2(z — +). 


Example 4.24 Determine the residues of the function 1/(1 +z’) at each of its poles in the finite z plane. 


Solution The function 1/(1 +z‘) has poles where 
1+27=0 
that is, at the points where 


7 = —| = eit 2mnj 
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with an integer. Recalling how to determine the roots of a complex number, these 
points are 


z= etiatnjni2 (n = 0, 1, 2. 3) 


that is 


nid 30/4 5ni/4  7Hj/4 
’ Ls > 


Z=e e e e 
or 
z= (1+jW2, C1 +2, C1 —pw2, 1 — D2 


To find the residue at the point z), we use (4.37), giving 


residue __,. Z-Zo 
= lim | —— 
at Zo 2929 1 + Zi) 


where z, is one of the above roots of z* = —1. It pays to use L’H6pital’s rule before 
substituting for a particular z,). This is justified since (z — Z))/(1 + z,) is of the indeter- 
minate form 0/0 at each of the four simple poles. Differentiating numerator and 
denominator gives 


lim Gee) = lim (4) 
zZ7Z9 1+z 229 Az 


since 4z° is not zero at any of the poles; 1/4z} is thus the value of each residue at z = Zp. 
Substituting for the four values (+1 + j)//2 gives the following: 


residue 1 


whe. = Te 
atz=(1+j)//2 ayn +i) (1 +j)/4, 
residue 1 . 
atz=(1-jy2 acy - jy" ( 4)/4, 
residue 1 eae 


at z= (-1 +j)/\2 7 A()\(-1 +5)" 


residue 1 : 
2 Pa a pcre weeps Eat Ol) car 
ae Ee ACE Lay 
Finding each Laurent series for the four poles explicitly would involve far more 
difficult manipulation. However, the enthusiastic reader may like to check at least one 
of the above residues. 


Next suppose that we have a pole of order two at z = Z). The function f(z) then has 
a Laurent series expansion of the form 


a_ a_ 
2 + 1 


(z-Z) 2-20 


f= 


+aj+a(z—2Z)+... 
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Again, we are only interested in isolating the residue a_,. This time we cannot use 
(4.37). Instead, we multiply f(z) by (z — z,)° to obtain 
(Z — 2p) flZ) = ag + a(Z — 2) + ao(Z— ZH) +... 


and we differentiate to eliminate the unwanted a_,: 
d 2 7 
ce [(Z — 2) f@)] = a1 + 2ag(z — 2) +... 


Letting z tend to z, then gives 
lim lee - =) =a, 
22 dz 

the required residue. 


We now have the essence of finding residues, so let us recapitulate and generalize. 
If f(z) has a pole of order m at z — Zp, we first multiply f(z) by (z — z))”. Ifm = 2, we 
then need to differentiate as many times as it takes (that is, m— | times) to make 
a_, the leading term, without the multiplying factor z — z). The general formula for 
the residue at a pole of order m is thus 


m-1 
—— lim {Sate - <M) (4.38) 
(m-1)! 29% | gz"! 

where the factor (m — 1)! arises when the term a_,(z — z))""' is differentiated m — 1 
times. This formula looks as difficult to apply as finding the Laurent series expansion 
directly. This indeed is often so; and hence experience and judgement are required. 
A few examples will help to decide on which way to calculate residues. A word 
of warning is in order here: a common source of error is confusion between the 
derivative in the formula for the residue, and the employment of L’ H6pital’s rule to 
find the resulting limit. 


Example 4.25 _ Determine the residues of 


= 7-2: 
(z+ 1) (2 +4) 


at each of its poles in the finite z plane. 


Solution Factorizing the denominator gives 


flz) = — 
(z+ 1)(z- 2j)(z+ 2j) 


so that f(z) has simple poles at z = 2j and z = —2j and a pole of order two at z =—1. 
Using (4.37), 
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. 2 
residue z -2z 


= hn z= —_—_—_——— 
atz=2j 232) (z+ 1) (z- 2j)(z+2)) 
—4-4j . 
a I +9) 
(2) + 1)'(4)) 
7 2. 
residue a eg 2-22 _ 
atz=—-2j 29-2 (z+ 1) (z- 2j)\(z + 2j) 
_ -4444 a, 
———-“l_ = 1(7-}) 


(-2j + 1)°(-4j) 
Using (4.38) with m = 2 we know that 


: 2 
residue 7 i lim da (z+ 2 22 
atz=-1l 1! 25-1 dz (Zea) (Zz +4) 


ake, (OD HE 22) HEH Oe?) — 8 
= lim ; = =—- 
a (z° +4) 25 


Example 4.26 Determine the residues of the following functions at the points indicated: 


@— €) ©(8) 6-9 © 5 G=-0 


(1+2°) P (z+ 1) 


Solution (a) Since 
a Lo 
@tly GHye-i 
and e* is regular at z= j, it follows that z=j is a pole of order two. Thus, from (4.38), 
sige 2c gia 56 e 
residue = lim —— | (ga pee, 
eae (z+j) @-)) 


=n {| e —— 
dz 


PACA Garay | (z+j)" 
_ (2jye'=2(2j)e’ ty 
= £ = f= (1 4je' 
(2j) 
Since e! = cos! + jsin1, we calculate the residue at z = j as 0.075 — j0.345. 


(b) The function [(sin z)/z*} has a pole at z = 0, and, since (sinz/z) > 1 as z > 0, 
(sin’z)/z? may also be defined as 1 at z= 0. Therefore, since 


the singularity at z= 0 must be a pole of order three. We could use (4.38) to obtain 
the residue, which would involve determining the second derivative, but it is easier 
in this case to derive the coefficient of 1/z from the Laurent series expansion 
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sin Z zz 

as ] -=4+=-— 
Zz 3! 5! 

giving 

snz_1 1 pelle oe 
aa 67 | 1207 

Z z 


Taking the cube of this series, we have 


Hence the residue at z= 0 is —}. 


(c) The function z‘/(z + 1)* has a triple pole at z = —1, so, using (4.38), 


residue = lim i (z+1)° z | - lim 1 (4 
zo-1 |“ dz (z+1)° | lis 


= lim 5x4x3z = 6(-1) = 6 


z>-1 


Residues are sometimes difficult to calculate using (4.38), especially if circular func- 
tions are involved and the pole is of order three or more. In such cases direct calculation 
of the Laurent series expansion using the standard series for sin z and cos z together with 
the binomial series, as in Example 4.26(b), is the best procedure. 


4.5.4 Exercises 


50 Determine the residues of the following rational ys sd y 
functions at each pole in the finite z plane: (c) 4] =e") (d) sin z (=n) 
2z+1 1 
a) (b) =—— = GH) 
z-z-2 z(1-z) (z +1) 
2 3 2 
32 +2 (d) goztzol 52 The following functions have poles at the points 
(z-1)(@ +9) z+4z indicated. Determine the order of the pole and the 
6 4.3 2 residue there. 
(e) Z +4z — (f) (44) 
(z-1) Zz (a) e087 (<=) 
z+l1 (h 3 +42 4 
(z-1)(z+3) 2432 +22 (in eal 


(z +1) (2° +4) 
51 Calculate the residues at the simple poles indicated 


(c) ue (z =nmt, n an integer) 
of the following functions: 2 ae a 


sin z 
(a) 982 @=0) (b) = sinz (¢ =e") (Hint: use lim,,,9(sin uy/u = 1 (u=z-—nn), after 
Zz Zitz +1 differentiating, to replace sin u by u under the limit.) 
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—=o Contour integration 


4.6.1 


Figure 4.25 
Partitioning of 
the curve C. 


Consider the definite integral 


| f(z) dz 


rill 

of the function f(z) of a complex variable z, in which z, and z, are a pair of complex 
numbers. This implies that we evaluate the integral as z takes values, in the z plane, 
from the point z, to the point z,. Since these are two points in a plane, it follows that to 
evaluate the definite integral we require that some path from z, to z, be defined. It is 
therefore clear that a definite integral of a complex function f(z) is in fact a line integral. 

Line integrals were considered in Section 3.4.1. Briefly, for now, a line integral in 
the (x, y) plane, of the real variables x and y, is an integral of the form 


| [P(x, y) dx + O(x, y) dy] (4.39) 
Cc 

where C denotes the path of integration between two points A and B in the plane. In the 
particular case when 


oP _ oQ 
Oy ox 


the integrand P(x, y)dx + O(x, y) dy is a total differential, and the line integral is 
independent of the path C joining A and B. 

In this section we introduce contour integration, which is the term used for evaluat- 
ing line integrals in the complex plane. 


(4.40) 


Contour integrals 


Let f(z) be a complex function that is continuous at all points of a simple curve C in the 
z plane that is of finite length and joins two points a and b. (We have not gone into great 
detail regarding the question of continuity for complex variables. Suffice it to say that 
the intuitive concepts described in Chapter 9 of Modern Engineering Mathematics for 
real variables carry over to the case of complex variables.) Subdivide the curve into n 
parts by the points z,, 2, ..., Z,_), taking z) = a and z, = b (Figure 4.25). On each arc 


joining z,_, to z, (k= 1,...,m) choose a point Z,. Form the sum 
y » 
nee =b 
2 rat set on) 
z, ae 
= Zy+1 
eer Zi» CUR 
Z 22 ZA 
Zi 
24=a 
0 x 
z plane 
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Example 4.27 


Figure 4.26 
Path of integration 
for Example 4.27. 


Solution 


S, =f(Z)(Z ~ Zo) t+ f(Z)(Z ~ 2) +... + fE Zn — Zn-1) 
Then, writing z,— z,_; = Az,, S,, becomes 


8,= > fE) Az, 


k=1 
If we let m increase in such a way that the largest of the chord lengths |Az,| approaches 


zero then the sum S,, approaches a limit that does not depend on the mode of subdivision 
of the curve. We call this limit the contour integral of f(z) along the curve C: 


| fiz) dz = lim YS fl, Ax (4.41) 


|Az,|70 = 
If we take z =x + jy and express f(z) as 


F(@) = ue, y) + ju, y) 
then it can be shown from (4.41) that 
| f(2) dz = | [u(x, y) + ju(x, y)](dx + j dy) 
(a (& 


or 


| f(z) dz -| [u(x, y) dx — v(x, y) dy] 


+ i| [v(x, y) dx + u(x, y) dy] (4.42) 


Both of the integrals on the right-hand side of (4.42) are real line integrals of the 
form (4.39), and can therefore be evaluated using the methods developed for such 
integrals. 


Evaluate the contour integral f.z7dz along the path C from —1 + j to 5 + j3 and com- 
posed of two straight line segments, the first from —1 + j to 5 +j and the second from 
5+jto5+ j3. 


z plane 


The path of integration C is shown in Figure 4.26. Since 


P= (xt jy =@?-y) + jaxy 


www.2Ofile.org 


4.6 CONTOUR INTEGRATION 319 


it follows from (4.42) that 


l= | zdz= | [(x° ay) dx - 2xy dy] +i| [2xy dx + (x? - y’) dy] 
c c c 


Along AB, y = | and dy = 0, so that 


5 2. 
ho=| w-Darsi| 2x dx 
-1 -1 
= [1x —x]', + jb7]§, = 36 + j24 
Along BD, x = 5 and dx = 0, so that 


3 3 
Tap = | —10y dy +) | (25 -y°) dy 


1 1 
= [-5y']i + j[25y — ty} 
=-40+j% 


Thus 


| Pee= toto = 5 4i24) +( 40 +j'*) =-44+j% 
GC 


Example 4.28 — Show that (z+ 1)dz=0, where C is the boundary of the square with vertices at z = 0, 
z=1+j0,z=1+jlandz=0+)l1. 


Solution The path of integration C is shown in Figure 4.27. 
Since z+ 1 = (x+ 1) +jy, it follows from (4.42) that 


. i=| erna=| (c+ Nax-ydy1 if [ydx+(x+1)dy] 
DO+) BU +)) . 
Along OA, y = 0 and dy = 0, so that 
1 
O ACI + j0) * ion | (x +1) dx = 3 
z plane ; 


Figure 4.27 Path = an 
of integration for Along AB, x = 1 and dx = 0, so that 


Example 4.28. 1 1 
a= | -revsi] 2dy =-}4+j2 


0 0 


Along BD, y = | and dy = 0, so that 


0 0 
to= | w+ Dare] dx =-3-j 


1 1 


Along DO, x = 0 and dx = 0, so that 
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4.6.2 


Theorem 4.1 


Proof 


0 0 
| -yarsi] dx = tj 
rl 1 


Thus 


| (z+ 1) dz = Toa tan + Lap + Lo =0 
C 


Cauchy’s theorem 


The most important result in the whole of complex variable theory is called Cauchy’s 
theorem and it provides the foundation on which the theory of integration with respect 
to a complex variable is based. The theorem may be stated as follows. 


Cauchy’s theorem 


If f(z) is an analytic function with derivative f’(z) that is continuous at all points 
inside and on a simple closed curve C then 


; fiz) dz = 0 


(Note the use of the symbol ¢, to denote integration around a closed curve, with the 
convention being that the integral is evaluated travelling round C in the positive or 
anticlockwise direction.) 


To prove the theorem, we make use of Green’s theorem in a plane, which was intro- 
duced in Section 3.4.5. At this stage a statement of the theorem is sufficient. 


If C is a simple closed curve enclosing a region A in a plane, and P(x, y) and Q(x, y) are 
continuous functions with continuous partial derivatives, then 


; (Pdi Od = | | (z . =) dx dy (4.43) 


Returning to the contour integral and taking 


f@ =u, y)+jue,y), z=x+hy 
we have from (4.42) 


; f(z) dz = ; (udx -vdy) + i (v dx +u dy) (4.44) 


Cc 


Since f(z) 1s analytic, the Cauchy—Riemann equations 


du_ a = dv __ du 


ox oy’ ox oy 


are satisfied on C and within the region R enclosed by C. 
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Since u(x, y) and v(x, y) satisfy the conditions imposed on P(x, y) and O(x, y) in 
Green’s theorem, we can apply (4.43) to both integrals on the right-hand side of (4.44) 
to give 


p fede |(-%- Sard i[|(H- 2) arg =o. 


R R 


by the Cauchy—Riemann equations. Thus 


; Az) dz = 0 


as required. 


end of theorem 


In fact, the restriction in Cauchy’s theorem that f’(z) has to be continuous on C can 
be removed and so make the theorem applicable to a wider class of functions. A revised 
form of Theorem 4.1, with the restriction removed, is referred to as the fundamental 
theorem of complex integration. Since the proof that f’(z) need not be continuous on 
C was first proposed by Goursat, the fundamental theorem is also sometimes referred 
to as the Cauchy—Goursat theorem. We shall not pursue the consequences of relaxa- 
tion of this restriction any further in this book. 

In practice, we frequently need to evaluate contour integrals involving functions such as 


A@=—-, 4@=—— 
Z=2 (z- 3) (24+2) 


74 


that have singularities associated with them. Since the function ceases to be analytic 
at such points, how do we accommodate for a singularity if it is inside the contour of 
integration? To resolve the problem the singularity is removed by deforming the contour. 

First let us consider the case when the complex function f(z) has a single isolated 
singularity at z = z) inside a closed curve C. To remove the singularity, we surround it 
by a circle y, of radius p, and then cut the region between the circle and the outer 
contour C by a straight line AB. This leads to the deformed contour indicated by the 
arrows in Figure 4.28. In the figure the line linking the circle y to the contour C is 
shown as a narrow channel in order to enable us to distinguish between the path A to 
B and the path B to A. The region inside this deformed contour is shown shaded in the 
figure (recall that the region inside a closed contour is the region on the left as we travel 
round it). Since this contains no singularities, we can apply Cauchy’s theorem and write 


YA 


x 


z plane 
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Example 4.29 


Solution 


z plane 


Figure 4.29 A circle 
of radius p, centred 
at the origin. 


; f(z) dz + | f(z) dz + p fe dz + ; f(z) dz =0 
e AB Y BA 


Since 


this reduces to 


; f(z) dz = ; f(z) dz (4.45) 
Cc yt 


with the + indicating the change of sense from clockwise to anticlockwise around the 
circle y. 


Evaluate the integral $,.dz/z around 


(a) any contour containing the origin; 


(b) any contour not containing the origin. 


(a) f(z) = 1/z has a singularity (a simple pole) at z = 0. Hence, using (4.45), the 
integral around any contour enclosing the origin is the same as the integral around 
a circle y, centred at the origin and of radius Py. We thus need to evaluate 


p be 
Zz 
7 


As can be seen from Figure 4.29, on the circle y 
z= pye’® (0 = O< 2n) 

so 
dz = jp, e'* de 


leading to 


2n, je 2m 
pla: -| ine a0- | jd =2nj 
~ 0 Pof& 0 


Hence if C encloses the origin then 


(b) If C does not enclose the origin then, by Cauchy’s theorem, 


c 7 


since 1/z is analytic inside and on any curve that does not enclose the origin. 


www.2Ofile.org 


Example 4.30 


Solution 


Example 4.31 


Solution 
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Generalize the result of Example 4.29 by evaluating 


pe 
oz 


where 7 is an integer, around any contour containing the origin. 


Ifn <0, we can apply Cauchy’s theorem straight away (or evaluate the integral directly) 
to show the integral is zero. If nm > 1, we proceed as in Example 4.29 and evaluate the 
integral around a circle, centred at the origin. Taking z= py e!° as in Example 4.29, we have 


A 2n ei? 
=|] io ao 
oe dene 
where fy 1s once more the radius of the circle. If n 4 1, 


a ae ee cas ee ee 
7g) en ig | S| ee ap 
i 7" | pr! gute JPo li - | l-n 


0 0 0 


since e*"/’ = | for any integer N. Hence 


; _9 (#1) 
cz 


In Examples 4.29 and 4.30 we have thus established the perhaps surprising result 
that if C is a contour containing the origin then 


dz _ [2nj (n=1) 
ex 0 (n any other integer) 


If C does not contain the origin, the integral is of course zero by Cauchy’s theorem. 


Evaluate the integral 


; dz 
perl] 
around any contour C containing the point z = 2 + j. 


The function 


1 
ged] 


f(z) = 
has a singularity (simple pole) at z= 2 + j. Hence, using (4.45), the integral around any 


contour C enclosing the point z = 2 + j is the same as the integral around a circle y 
centred at z= 2 +j and of radius p. Thus we need to evaluate 


; dz 
yer ar] 
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z plane 


Figure 4.30 A circle 
of radius p centred at 
2+]. 


Figure 4.31 
Deformed contour 
for n singularities. 


Example 4.32 


Solution 


As can be seen from Figure 4.30, on the circle y 
z=(2+j)+pe” (0<@<2n) 
dz =jpe de 
leading to 


20. 0 2n 
p | ios a9 = | jdO = 2nj 
go es 0 Pe 


0 


Hence if C encloses the point z = 2 +] then 


; dz = Inj 
e272] 


Compare this with the answer to Example 4.29. 


So far we have only considered functions having a single singularity inside the 
closed contour C. The method can be extended to accommodate any finite number of 
singularities. If the function f(z) has a finite number of singularities at z= z,, z,,..., z, 


2 “n? 


inside a closed contour C, then we can deform the latter by introducing n circles ,, %, 
., Y, to surround each of the singularities as shown in Figure 4.31. It is then readily 
shown that 


; f(z) dz = ; f(z) dz + ; f(iz)dz+...4+ ; f(z) dz (4.46) 
€ YN Y2 


Tn 


Evaluate the contour integral 


zdz 
et 1)(z + 2j) 


where C is 


(a) any contour enclosing both the points z = | and z = —2); 


(b) any contour enclosing z = —2j but excluding the point z = 1. 


The function 
_ Zz 

We ies 
has singularities at both z= | and z = —2). 


(a) Since the contour encloses both singularities, we need to evaluate the integrals 
around circles y, and y, of radii p, and p, surrounding the points z= | and z = —2j 
respectively. Alternatively, we can resolve f(z) into partial fractions as 

4(1-j2)  4£(44+2j 
(i 


z-l z+2j 
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and consider 
dz : dz : dz 
=f tan A Hasan} SS =nt+h 
-(2-1)(2-2j) 3 oz-1 3 o2t2j 


The integrand of J, has a single singularity at z = 1, and we simply need to 
evaluate it around the circle y, of radius p, about z= | to give 


[= 2nj 


Similarly, /, has a single singularity at z=—2j, and we evaluate it around the circle 
Y, to give 


1,=2nj 
Then 
T= 3 (1 —j2)2nj + $ (4 + j2)2nj = 2nj(2 - j4) 
Thus if the contour C contains both the singularities then 
7 ae = 2njtz — i) 
(b) Ifthe contour C only contains the singularity z = —2] then 


zdz ly 35 
seers: = J = 2s ie) 
(z- 1)(z+2j) 5 


In Examples 4.29—4.32 we can note some similarity in the answers, with the common 
occurrence of the term 27j. It therefore appears that it may be possible to obtain some 
general results to assist in the evaluation of contour integrals. Indeed, this is the case, 
and such general results are contained in the Cauchy integral theorem. 


Theorem 4.2 Cauchy integral theorem 


Let f(z) be an analytic function within and on a simple closed contour C. If z) is any 
point in C then 


; fi. dz = 2nj f(z) (4.47) 


If we differentiate repeatedly n times with respect to z under the integral sign then it 
also follows that 


; Ae) — az = Hl f%%2,) (4.48) 


(z = cee 


end of theorem 


Note that (4.48) implies that if f(z) exists at z = z, so does f(z) for all n, as predicted 
earlier in the observations following Example 4.16. 
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Example 4.33 


Solution 


Example 4.34 


Solution 


Evalute the contour integral 


; 2z 
a 
¢ @-Y)E+2)e+)j) 


where C is a contour that includes the three points z = 1, z=—2 and z=—j. 


Since 


fiz) = 7 


(z- 1)(z+ 2)(z4+j) 
has singularities at the points z= 1, z=—2 and z =—j inside the contour, it follows from 
(4.46) that 


; f(z) dz = ; f(z) dz + ; f(z) dz + f f(z) dz (4.49) 
c n 1 13 


where ¥;, 7 and ¥; are circles centred at the singularities z = 1, z = —2 and z=~—j 
respectively. In order to make use of the Cauchy integral theorem, (4.49) is written as 


; fee alice aera +4 12z/[(zZ-DErpt g, 
Cc ‘1 Y2 


z-1l z+2 


4 {2z/[(z- 1l)(z+2) bas 
3 


z+] 
= h@) dz + £@) dz + (2) dz 
1,271 y,2+2 y,2+5 
Since f,(z), f(z) and f(z) are analytic within and on the circles y,, y, and y; respectively, 
it follows from (4.47) that 


; flz) dz = 2njLfiC) + fA(-2) +A) 


= 2Tj pat ee + een: ee + - =2j - 
Al) (-3)(-249) Gio I-42) 
so that 


2z dz ai 
aie= 1)(z+2)(z+j) 


Evaluate the contour integral 


; z 5 dz 
c(z-1) 


where the contour C encloses the point z = 1. 


Since f(z) = z‘/(z — 1) has a pole of order three at z = 1, it follows that 


; feyie=4 af as 
c y(z- 1) 
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where y is a circle centred at z = 1. Writing f,(z) = 2‘, then 


; feyie= 4 AD az 
c 7 (z- 1) 


and, since f\(z) is analytic within and on the circle y, it follows from (4.48) that 


; fiz) dz = 2nj ee) 


so that 


4.6.3 Exercises 


Evaluate J.(z* + 3z) dz along the following contours 
C in the complex z plane: 


(a) the straight line joining 2 + j0 to 0 + j2; 

(b) the straight lines from 2 + j0 to 2 + j2 and then 
to 0+ 42; 

(c) the circle |z| =2 from 2 + j0 to 0+ j2 in an 
anticlockwise direction. 


Evaluate ,(5z* — z’ + 2) dz around the following 
closed contours C in the z plane: 


(a) the circle |z| = 1; 

(b) the square with vertices at 0 + j0, 1 +0, 
1+jl and0+jl; 

(c) the curve consisting of the parabolas y =x? from 
0+j0tol+jl andy =x from 1+jto0+j0. 


Generalize the result of Example 4.30, and show that 


; dz _ a 
Cc (z= Zo)" 0 


where C is a simple closed contour surrounding 
the point z = Zp. 


(n= 1) 
(n #1) 


Evaluate the contour integral 


; dz 
z-4 
é 


where C is any simple closed curve and z = 4 is 


(a) outside C (b) inside C 
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58 


SY) 


= nj(12z’).- 


Using the Cauchy integral theorem, evaluate the 
contour integral 


2z dz 
ss (2z - 1)(z+2) 


where C is 


(a) the circle |z|=1 
(b) the circle |z| =3 


Using the Cauchy integral theorem, evaluate the 
contour integral 


—__52zdz 
e (z+ 1)(z-2)(2+4)j) 
where C is 


(a) the circle |z| =3 
(b) the circle |z| =5 


Using the Cauchy integral theorem, evaluate the 
following contour integrals: 


(a) ; 2: 
¢ 2z+1) 


where C is the unit circle |z| = 1; 


b) ; 2 a 
c (z- 1)(z+2) 


where C is the circle |z| = 3. 
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4.6.4 


The residue theorem 


This theorem draws together the theories of differentiation and integration of a complex 
function. It is concerned with the evaluation of the contour integral 


l= ; f(z) dz 


where the complex function f(z) has a finite number n of isolated singularities at z,, 
Z,...,Z, mside the closed contour C. Defining the contour C as in Figure 4.31, we 
have as in (4.46) that 


— ; f(z) dz = ; f(z) dz + ; f(z) dzt+...+ ; f(z) dz (4.46) 
[a Nn Y2 Vn 


If we assume that f(z) has a pole of order m at z = z, then it can be represented by the 
Laurent series expansion 


a? a® . | 
, ey Sle asi’ : = + a a (2-2) + ree + a(z-2z,)" + _ 
ZZ; =; 


Az) = 


valid in the annulus 7, < |z — z,| < R,. If the curve C lies entirely within this annulus 
then, by Cauchy’s theorem, (4.46) becomes 


Is ; f(z) dz - f(z) dz 
c Yj 


i 


Substituting the Laurent series expansion of f(z), which we can certainly do since we 
are within the annulus of convergence, we obtain 


(i) (@) 
a. a. 
i f(z) dz = ; at tap ta (z-z) +... 
= 
1; Y 


i dz dz 
= ang —— +... 4.08) —+ao dz 
Yj (Zz - Z;) 7,7 7i Yj 


Using the result from Exercise 55, all of these integrals are zero, except the one 
multiplying a“), the residue, which has the value 27j. We have therefore shown that 


; fiz) dz = 2nja“ = 2nj x residue at z = z, 
% 
This clearly generalizes, so that (4.46) becomes 


i= ; f(z) dz = 2nj ¥ (residue at z = z,) 
Cc 


i=l 


= 2m) x (sum of residues inside C) 


Thus we have the following general result. 
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Example 4.35 


Solution 


Example 4.36 


Solution 
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The residue theorem 


If f(z) is an analytic function within and on a simple closed curve C, apart from 
a finite number of poles, then 


; f(z) dz = 2nj x [sum of residues of f(z) at the poles inside C] 
(@ 


end of theorem 


This is quite a remarkable result in that it enables us to evaluate the contour integral 
. f(z) dz by simply evaluating one coefficient of the Laurent series expansion of f(z) at 
each of its singularities inside C. 


Evaluate the contour integral ¢, dz/[z(1 + z)] if C is 


(a) the circle |z| = 3; (b) the circle |z| = 2. 


The singularities of 1/[z(1 + z)] are at z= 0 and —1. Evaluating the residues using (4.37), 
we have 


residue _ ij de 
= limz ———~ = 
atz=0 :50 z(1+z) 
residue _ im eaet) | re 
atz=-l 2-1 z(1+z) 


(a) IfCis|z|= ; then it contains the pole at z= 0, but not the pole at z=—1. Hence, 
by the residue theorem, 


Oe gs 2mj X (residue at z = 0) = 27) 
nete+) 


(b) If Cis|z|=2 then both poles are inside C. Hence, by the residue theorem, 


a a 
p ey =m 1) =0 


3 2 
Evaluate the contour integral ; 2 42 dz where C is 
c 2 t4z 


(a) |z|=1 (b) |z| =3 
The rational function 
gas teo 1 


2+4z 
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has poles at z= 0 and +2). Evaluating the residues using (4.37) gives 


: 3 2 
residue ~ tim = 2 tz-1)_ | 


atz=0 230 2(z’ +4) 7 


‘: ci 4 2 
residue =A) - =1) ' 
= lim @ 2D -z4+z-1)_ 3 ai 3 


atz=2j 9-923 z(z—2j)\(z+2j) §*° 4 
residue _ li @P2)(2 -2 +2-1)_ 3 3: 

— 4 = tm : ; mt eee a 
atz=—2j 25-2) 2z(z — 2j)(z + 2)) 


(Note that these have been evaluated in Exercise 50(d).) 


(a) IfCis|z|=1 then only the pole at z = 0 is inside the contour, so only the residue 
there is taken into account in the residue theorem, and 


3 2: 
; goztel dz = 2nj(-1) = -4nj 
c 2t4z 


(b) If Cis |z| = 3 then all the poles are inside the contour. Hence, by the residue 
theorem, 


z-z4+z-1 i By. Se 3 : 
a eel dz= 27} ( 4 at J 3 al) = 2Tj 
c 2zt4z 


Example 4.37 — Evaluate the contour integral 


dz 
z(z+2z+2) 


where C is the circle |z| = 3. 


Solution The poles of 1/z*(z* + 2z + 2) are as follows: a pole of order three at z = 0, and two 
simple poles where z* + 2z + 2 = 0, that is at z= —1 +j. All of these poles lie inside the 
contour C. 
From (4.38), the residue at z = 0 is given by 


i 1a 1 2 ee ed Sez | on ad -(z+1) 
iss | |= lim 5 | — 2} = lim | > 7 
2390 2! q,? | 2°4+274+2 230° dz | (z +2z+2) 230 dz | (2° +2z+2) 
= jy HS Ee yt 12" $224 2)2242) i 

290 (2° +22 +2)" i 


From (4.37), the residue at z =—1 —j is 
; : 1 a 1 
in aaa S—§ii —_— = lin ——— 
earls) Z(zt+14j)(zt+1-j) 7eb z(z4+1-J) 
se ea ee a carer care 
Clea). Gaay2ay eens) 
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z plane 


Figure 4.32 
The closed contour for 
evaluating f°. f(x) dx. 
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using (1 +j)' = 14+ 3) + 377+)? =—2 + 2). Hence 


residue 1 L-j _ 


= be eS ol — 1] +] 
atz=-l-j ‘*-1-j mee) at )) 
Also, using (4.37), 
residue _= lim (¢+1-j)- 1 
atz=-l4+j -o-1j z(z+14+j)\@e129) 


which is precisely the complex conjugate of the residue at z=—1 — j. Hence we can take 
a short cut with the algebra and state the residue as }(—1 — }). 
The sum of the residues is 


L+l-14j)+iC-1-j) = 0 
so, by the residue theorem, 


; ——% _ = ono) = 0 


z(z+2z+2) 


Evaluation of definite real integrals 


The evaluation of definite integrals is often achieved by using the residue theorem 
together with a suitable complex function f(z) and a suitable closed contour C. In this 
section we shall briefly consider two of the most common types of real integrals that 
can be evaluated in this way. 


Type 1: Infinite real integrals of the form J”, f(x) dx where f(x) is a 
rational function of the real variable x 


To evaluate such integrals we consider the contour integral 


; f(z) dz 


where C is the closed contour illustrated in Figure 4.32, consisting of the real axis from 
—R to +R and the semicircle I, of radius R, in the upper half z plane. Since z = x on the 
real axis, 


; fz) dz = | fx) dx + | fiz) dz 


-R rT 


Then, provided that limg.,.. fr f(z) dz = 0, taking R > © gives 


; f(z) dz = | f(x) dx 


On the semicircular path I, z= Re! (0<O0< 


dz = jRe!® do 


} 1), giving 
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Example 4.38 


Solution 


and 


| fiz) dz = | f(Re")jRe" do 


0 


For this to tend to zero as R  o, |f(Re?®)| must decrease at least as rapidly as R°, 
implying that the degree of the denominator of the rational function f(x) must be at least 
two more than the degree of the numerator. Thus, provided that this condition is 
satisfied, this approach may be used to calculate the infinite real integral [~, f(x) dx. 
Note that if f(x) is an even function of x then the same approach can also be used to 
evaluate fj f(x) dx, since if f(x) is even, it follows that 


| fix) dx = 2 | fx) dx 


Using contour integration, show that 
dx 
fei 
_o (x + 4) 


Consider the contour integral 


dz 
I = 
(2° +4) 


where C is the closed semicircular contour shown in Figure 4.32. The integrand 
1/(z? + 4) has poles of order two at z = +2]. However, the only singularity inside the 
contour C is the double pole at z = 2j. From (4.38), 


residue __ |. im 42 AG So 1 
atz=2j a8)! lier (z + 2i)° 
-2 2 ied 


m —— = =s 5] 
#92) (z+2j) (4j)) 


so, by the residue theorem, 


; dz 
oe ay 


Since 


; dz -| dx | dz 
(2 +4yY p(et4y Jp (2 4+4y 


letting R — , and noting that the second integral becomes zero, gives 


; dz 
als +47 


= 2nj(—33i) = 16% 


co 


_ dx a 
=| +a 
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Note that in this particular case we could have evaluated the integral without using 
contour integration. Making the substitution x = 2tan@, dx = 2sec’0 d@ gives 


a n/2 2 n/2 
| aaa. = | ae Tay! = i cos 6 dé = <[} sin 20 + ol". =in 
awlX =F -n/2 sec 


-1/2 


Type 2: Real integrals of the form J = J("G(sin 0, cos 8) dO where G is 
a rational function of sin@ and cos 0 


We take z = e!°, so that 


dz=jede, or de= 2% 


JZ 


On substituting back, the integral 7 becomes 


z plane 


Figure 4.33 The 
unit-circle contour I= f(z) dz 
for evaluating c 


Je7G¢sin 6, cos 6) dé. where C is the unit circle |z| = 1 shown in Figure 4.33. 


Example 4.39 — Using contour integration, evaluate 
2n 
l= _ do 
» 2+ Cos é 


Solution Take z=e!®, so that 


cos = 1(2+4), dé= = 
Zz 


On substituting, the integral becomes 


Fe ; dz _2 ; dz 
oJZ[2+ (zt liz)]) J c 2+4z4+1 
where C is the unit circle |z| = 1 shown in Figure 4.33. The integrand has singularities at 
z+4z7+1=0 


that is, at z=—2 + /3. The only singularity inside the contour C is the simple pole at 
z=—2+ /3. From (4.37), 


residue at z=—2 + \3 


= lim 2 —EE— =? -1 
z j (z24+2- J3)(z+2+3)} j2/3 jy3 


www.2Ofile.org 


334 FUNCTIONS OF A COMPLEX VARIABLE 


60 


61 


62 


63 


so, by the residue theorem, 


I= 2ni( | = 22 
jy3 \3 


Thus 


20 
d@__ «2a 
; 2+cos@ {3 


4.6.6 Exercises 


Evaluate the integral 


zdz 
is z+) 


where C is 


(a) the circle |z| = S (b) the circle |z| =2 


Evaluate the integral 
2 . 
Z3jZ= 2 re 
. 2492 
where C is 


(a) the circle |z| = 1 (b) the circle |z| =4 


Calculate the residues at all the poles of the function 
fz) = (2 +2 )(z +4) 
(2+ 1)(z' +6) 


Hence calculate the integral 


; f(z) dz 


where C is 


(a) the circle |z| =2 
(c) the circle |z|=4 


(b) the circle |z—j| = 1 


Evaluate the integral 


dz 
oz (tz) 


where C is 


(a) the circle |z| = 5 (b) the circle |z| = 2 


64 


Using the residue theorem, evaluate the following 
contour integrals: 


(a) ; (32° +2) dz 
o @-1W"+4) 


(i) the circle |z-2| =2 


where C is 
(ii) the circle |z| = 4 


(b) ; (z’ - 2z) dz 
o 241) +4) 


(i) the circle |z| = 3 


where Cis 4° ; 
(ii) the circle |z +j| = 2 


) dz 
o (241) (z- 1)(z- 2) 


(i) the circle |z| = 4 
(ii) the circle |z +1] = 1 


(iii) the rectangle with vertices 
at tj, 3+j 


where C is 


(d) ; giles Mae. 
c (2 -4)(z4+1)° 
(i) the circle |z| =} 


(ii) the circle |Z +3/=2 


where C is 
(iii) the triangle with vertices 


at -$+j, -3-j, 3+j0 
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65 Using a suitable contour integral, evaluate the - 2 
2 ; x dx 
following real integrals: ( ae Te a Ae 
~ +1) (+ 2x42) 
dx dx ° 
a) = O | as (g) (hy | 
xX +x4+1 a DD 3- Su, te ox +1 
d 
(©) | [ee (i) 
o (xX + 1)(x +4) (44x45) vere 
20 20 
cos 30 4 dé __cosO __ 
@ i 5-4 cos 0 (°) i 5+4 sin@ wf 325 cos 0°" 


Uy ae atedltetetalarearlyeiteciielaa analysing AC circuits 


In the circuit shown in Figure 4.34 we wish to find the variation in impedance Z and 

- admittance Y as the capacitance C of the capacitor varies from 0 to c. Here 
1_1, jac, yet 
Z 


Z R 
Writing 
-— 

Figure 4.34 1 _1+j@Cr 
AC circuit of Z R 
Section 4.7. 

we clearly have 

ae ar (4.50) 
1+j@CR 


Equation (4.50) can be interpreted as a bilinear mapping with Z and C as the two vari- 
ables. We examine what happens to the real axis in the C plane (C varies from 0 to © 
and, of course, is real) under the inverse of the mapping given by (4.50). Rearranging 
(4.50), we have 


ee (4.51) 
JORZ 
Taking Z=x + jy 
Ray Re KR. ay = Ry ix (4.52) 
jOR(x+jy) @R(y-jx) oR(x +y) 
Equating imaginary parts, and remembering that C is real, gives 
O=xr+y—Rx (4.53) 


which represents a circle, with centre at (}R, 0) and of radius }R. Thus the real axis in 
the C plane is mapped onto the circle given by (4.53) in the Z plane. Of course, C is 
positive. If C = 0, (4.53) indicates that Z = R. The circuit of Figure 4.34 confirms 
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Figure 4.35 Mapping 
for the impedance Z. 


C plane Z plane 


Figure 4.36 Mapping 


for the admittance Y. Cee 


C increasing 


that the impedance is R in this case. If C  - then Z > 0, so the positive real axis in 
the plane is mapped onto either the upper or lower half of the circle. Equating real parts 
in (4.52) gives 
C= —_?~ 
a(x + y’) 
so C > 0 gives y < 0, implying that the lower half of the circle is the image in the 
Z plane of the positive real axis in the C plane, as indicated in Figure 4.35. A diagram 
such as Figure 4.35 gives an immediate visual impression of how the impedance 


Z varies as C varies. 
The admittance Y= 1/Z is given by 


y=14joc 
R 


which represents a linear mapping as shown in Figure 4.36. 


4.8 Engineering application: (Sema elaiteilem ile 


In this section we discuss two engineering applications where use is made of the 
properties of harmonic functions. 


4.8.1 A heat transfer problem 


We saw in Section 4.3.2 that every analytic function generates a pair of harmonic 
functions. The problem of finding a function that is harmonic in a specified region 
and satisfies prescribed boundary conditions is one of the oldest and most important 
problems in science-based engineering. Sometimes the solution can be found by means 
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Temperature 0 °C 


Temperature 100°C 


Figure 4.37 
Schematic diagram of 
heat transfer problem. 


r=0.3R 


Figure 4.38 
The mapping 
w = (z- 3)/3z- 1). 
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of a conformal mapping defined by an analytic function. This, essentially, is a con- 
sequence of the ‘function of a function’ rule of calculus, which implies that every 
harmonic function of x and y transforms into a harmonic function of wu and v under the 


mapping 
w=utjo=fx+jy)=/@ 


where f(z) is analytic. Furthermore, the level curves of the harmonic function in the 
z plane are mapped onto corresponding level curves in the w plane, so that a harmonic 
function that has a constant value along part of the boundary of a region or has a zero 
normal derivative along part of the boundary is mapped onto a harmonic function with 
the same property in the w plane. 

For heat transfer problems the level curves of the harmonic function correspond to 
isotherms, and a zero normal derivative corresponds to thermal insulation. To illustrate 
these ideas, consider the simple steady-state heat transfer problem shown schematic- 
ally in Figure 4.37. There is a cylindrical pipe with an offset cylindrical cavity through 
which steam passes at 100°C. The outer temperature of the pipe is 0 °C. The radius of 
the inner circle is + of that of the outer circle, so by choosing the outer radius as the 
unit of length the problem can be stated as that of finding a harmonic function 7(x, y) 
such that 


oT + dT = (0) 
ox” oy 
in the region between the circles |z| = 1 and |z— 0.3] = 0.3, and T= 0 on|z|= 1 and 
T = 100 on |z — 0.3| = 0.3. 
The mapping 


_ z-3 


~ 3z-1 


transforms the circle |z| = 1 onto the circle | w]| = 1 and the circle |z — 0.3| = 0.3 onto 
the circle |w| = 3 as shown in Figure 4.38. Thus the problem is transformed into the 
axially symmetric problem in the w plane of finding a harmonic function 7(u, v) such 
that 7(u, v) = 100 on |w| = 1 and 7(u, v) = 0 on |w|=3. Harmonic functions with such 
axial symmetry have the general form 


Tu, v)= Aln(u’ +") +B 


where A and 8& are constants. 
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Here we require, in addition to the axial symmetry, that 7(u, v) = 100 on uw? + v* = 1 
and 7(u, v) =0 on uw? +v* = 9. Thus B = 100 and A =—1001n9, and the solution on the 
w plane is 


2,2 
Twa) = 100[1 ite tv )] 


We need the solution on the z plane, which means in general we have to obtain uv and 
v in terms of x and y. Here, however, it is a little easier, since w° + v? =|w|’ and 


wee | Zoe iSrentT jC alesse 
3z-1]  |3z-1)? (3x-1)?+9y 
Thus 
TG, yy = ee 6 — ine -39 + 7] -n[Gx- 1% 4 97} 


In9 


4.8.2 Current in a field-effect transistor 


The fields (£,, £,) in an insulated-gate field-effect transistor are harmonic conjugates 
that satisfy a nonlinear boundary condition. For the transistor shown schematically in 
Figure 4.39 we have 


aE, _9E, By _ 88, 
ox oy’ Oy ox 


with conditions 


E.=0 on the electrodes 


E,(E,+ =) = cue on the channel 
“oh 2MEE, 


J 


E34 as x—>-0 (O<y</h) 


E,> +— as x70 (0<y<h) 


where Vy is a constant with dimensions of potential, / is the insulator thickness, / is the 
current in the channel, which is to be found, wu, €) and €, have their usual meanings, and 
the gate potential V, and the drain potential VY, are taken with respect to the source 
potential. 


Figure 4.39 

(a) Schematic diagram 
for an insulated-gate 
field-effect transistor; 
(b) an appropriate 
coordinate system for ——_—_—_—_—____— =. == $ —$_________% 
the application. Source electrode Channel Drain electrode 


Gate electrode 
OO OST 


(a) (b) 
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The key to the solution of this problem is the observation that the nonlinear boundary 
condition 


26,(z, +2) = - : 
i nee 3 HEE, 


contains the harmonic function (now of E, and £,) 


H{Ey E,) = 2E,(E, +72) 


A harmonic conjugate of H is the function 


V 2 
GE, E,) = (4, + =) -E 


Since E, and E,, are harmonic conjugates with respect to x and y, so are G and H. Thus 
the problem may be restated as that of finding harmonic conjugates G and H such that 


H=0_ on the electrodes 


H=- i on the channel 
0; 


2 2 
G— (2%) as x30 (0O<y</h) 


_y 
Go (BH) as x5 O<y<h) 


Using the sequence of mappings shown in Figure 4.40, which may be composed into 
the single formula 


bz 2 
_ae--a 
w= a ae 
ae -1 
where a =e” and b= 1/h, the problem is transformed into finding harmonic-conjugate 


functions G and H (on the w plane) such that 


H=0 on v=0 (u>0) (4.54) 
H=-—— om 920 G=0) (4.55) 
HEE, 
= 2 
em (a at w= eb (4.56) 
= 2 
ce (Ate) at w=1 (4.57) 


The conditions (4.54), (4.55) and (4.57) are sufficient to determine H and G completely 


i= _Larg(w) 


TLE E, 


G= T|n|w| (a4 Fee Hs) 
TEE, h 
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Figure 4.40 y 
Sequence of mappings 
to simplify the 
problem. z plane 
zoOztsh 
Zz Tzth 
zoe 
z7>z-1 


z— az(a2 -1) 


zo liz 


z31l-z 


sV 


w plane 


while the condition (4.56) determines the values of J 
p= 422% 2, -2V,.4+ VV; 
Lh i 


This example shows the power of complex variable methods for solving difficult 
problems arising in engineering mathematics. The following exercises give some 
simpler examples for the reader to investigate. 
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4.8.3 Exercises 


Show that the transformation w = 1/z, w=u+jv, 
z=x-+jy, transforms the circle x? +’ = 2ax in the 
z plane into the straight line w = 1/2a in the w plane. 
Two long conducting wires of radius a are placed 
adjacent and parallel to each other, so that their 
cross-section appears as in Figure 4.41. The 

wires are separated at O by an insulating gap of 
negligible dimensions, and carry potentials +V, 
as indicated. Find an expression for the potential 
at a general point (x, y) in the plane of the cross- 
section and sketch the equipotentials. 


#V 


Figure 4.41 Conducting wires of Exercise 66. 


Find the images under the mapping 


-2tl 
l-z 
z=x+yy, of 


(a) the points A(-1, 0), B(O, 1), C(3, =) and 
DG, 0) in the z plane, 

(b) the straight line y = 0, 

(c) the circle x? +)" = 1. 


Illustrate your answer with a diagram showing the 
z and w planes and shade on the w plane the region 
corresponding to x* +? < 1. 

A semicircular disc of unit radius, [(x, y): 
x+y <1, y= 0], has its straight boundary at 
temperature 0 °C and its curved boundary at 100 °C. 
Prove that the temperature at the point (x, y) is 


7 = tar! (—2- ;) 
Tt L-x =y 


(a) Show that the function 


G(x, y) = 2x(1 — y) 
satisfies the Laplace equation and construct 
its harmonic conjugate H(x, y) that satisfies 
(0, 0) = 0. Hence obtain, in terms of z, where 
z=x+Jy, the function F such that W = F(z) 
where VW=G+jH. 


(b) Show that under the mapping w = Inz, the 
harmonic function G(x, y) defined in (a) is 
mapped into the function 
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70 


G(u, v) = 2e"cosv — e* sin2v 
Verify that G(u, v) is harmonic. 


(c) Generalize the result (b) to prove that under 
the mapping w = f(z), where f’(z) exists, a 
harmonic function of (x, y) is transformed 
into a harmonic function of (uw, v). 


Show that if w = (z+ 3)(z-3), w=u+tjpu, 
z=x+jy, the circle uv’ + v* =k? in the w plane 
is the image of the circle 
2 
V+V+6—Sxt+9=0 (#1) 
1-k 
in the z plane. 
Two long cylindrical wires, each of radius 

4 mm, are placed parallel to each other with their 
axes 10 mm apart, so that their cross-section 
appears as in Figure 4.42. The wires carry potentials 
+V, as shown. Show that the potential V(x, y) at the 
point (x, y) is given by 


y= 2 (infor +3)?+y"]—In[(e— 3" +y°]} 
In 4 


> 
x 


Figure 4.42 Cylindrical wires of Exercise 69. 


Find the image under the mapping 


2 itt) 


~  Ll4z 
Z=x+jy,w=utyjp, of 


(a) the points A(1, 0), B(O, 1), C(O, —1) in the 
z plane, 

(b) the straight line y = 0, 

(c) the circlex’?+)°=1. 


A circular plate of unit radius, [(x, y): 2° +)” < 1], 
has one half (with y > 0) of its rim, x° +)” = 1, at 

temperature 0 °C and the other half (with y < 0) at 
temperature 100 °C. Using the above mapping, prove 
that the steady-state temperature at the point (x, y) is 


22 
r= 10 tan '( ear —— } 
Tt 2y 
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71 The problem shown schematically in Yh 
Figure 4.43 arose during a steady-state heat - D[C a 
transfer investigation. T is the temperature. 
By applying the successive mappings F A x 
=]. SE 4 
w=zt z 
vp 
2 -l iL 2 s 
oe 
Gi. IF’ ED’ C’[B’ BBA BH. 
T=0 
Figure 4.43 Schematic representation of yA 
Exercise 71. 
z= 20 ; w=Inz, -1 1 x 
z-j4 Se 
show that the temperature at the point (x, y) in the G.. FE Be i Aco 
shaded region in the figure is given by vo 
zZ— Vv A 
50 x+(4tyy -1 1 
Te, y)= — In u 
In3 fered : = . 
Cs D’' RE | FG A Ba 
72 The functions 
Figure 4.44 Mappings of Exercise 72. 
_ 1 _ztil 
w=zt+-, w= 
Zz z-l 


perform the mappings shown in Figure 4.44. A long 
bar of semicircular cross-section has the temperature YA 
of the part of its curved surface corresponding to 
the arc PQ in Figure 4.45 kept at 100 °C while the Q P 
rest of the surface is kept at 0 °C. Show that the 
temperature 7 at the point (x, y) is given by 

_ 100 


: a [arg ae arg(z’ zo) Figure 4.45 Cross-section of bar of Exercise 72. 


4.9 Review exercises (1-24) 


1 Find the images of the following points under the 2 Under each of the mappings given in Review 
mappings given: exercise 1, find the images in the w plane of the 


! traight li 
(a) z=14+j under w=(1+j)z+j oS nua 


(b) z=1-j2 under w=j3z+j+l (a) y= 2x 
(@) geil under w= i(1—j)z+}(1+j) (b) xt+y=1 
(d) z=j2 under w=i(1-j)z+$(1+)) in the z plane, z =x + jy. 
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The linear mapping w = az + B, where wand f are 
complex constants, maps the point z = 2 —j in the 
z plane to the point w = | in the w plane, and the 
point z = 0 to the point w =3 +j. 


(a) Determine a and fp. 

(b) Find the region in the w plane corresponding to 
the left half-plane Re(z) < 0 in the z plane. 

(c) Find the region in the w plane corresponding to 
the circular region 5|z| < 1 in the z plane. 

(d) Find the fixed point of the mapping. 


Map the following straight lines from the 

z plane, z=x + jy, to the w plane under the 

inverse mapping w = j/z: 

(a) x=y+1 

(b) y=3x 

(c) the line joining A(1 + j) to B(2 +j3) in the 
z plane 

(@) y=4 


In each case sketch the image curve. 


Two complex variables w and z are related by the 
mapping 


Sketch this mapping by finding the images 
in the w plane of the lines Re(z) = constant and 
Im(z) = constant. Find the fixed points of the 


mapping. 


The mapping 


takes points from the z plane to the w plane. Find 
the fixed points of the mapping, and show that the 
circle of radius r with centre at the origin in the 

z plane is transformed to the ellipse 


2 <7) re Ne r 
CokGaia 
r-1l r+1 


in the w plane, where w = u + jv. Investigate what 
happens when r = 1. 


Find the real and imaginary parts of the complex 
function w =z’, and verify the Cauchy-Riemann 
equations. 
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Find a function v(x, y) such that, given 
u(x, y) =xsinx coshy — ycosx sinhy 


f(z) =u + jv is an analytic function of z, f(0) = 0. 


Find the bilinear transformation that maps the three 
points z= 0, j and mil + j) in the z plane to the 
three points w = co, —j and 1 —] respectively in the 
w plane. Check that the transformation will map 


(a) the lower half of the z plane onto the upper 
half of the w plane 

(b) the interior of the circle with centre z = iz 
and radius } in the z plane onto the half-plane 
Im(w) < —1 in the w plane. 


Show that the mapping 
a 
— +— 
BENG ac 


where z = x + jy and €= Re! maps the circle 
R=constant in the ¢ plane onto an ellipse in the 
z plane. Suggest a possible use for this mapping. 


Find the power series representation of the 
function 


1 
ees 


in the disc |z| < 1. Deduce the power series for 
1 
(l+zy 


valid in the same disc. 


Find the first four non-zero terms of the Taylor 
series expansion of the following functions about 
the point indicated, and determine the radius of 
convergence of each: 


1 


l-z = = 
OO 6) 10) ee 
© e€=) 


Find the radius of convergence of each Taylor 
series expansion of the following function about the 
points indicated, without finding the series itself: 


at the points z= 1,-1, 1+], 1 Fis and 2 + j3. 
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14 


tS 


16 


7 


18 


iS) 


20 


Determine the Laurent series expansion of the 
function 
1 
2 
(ee ae ye 


about the points (a) z = 0 and (b) z= 1, and 
determine the region of validity of each. 


f@) = 


Find the Laurent series expansion of the function 
oa 1 
2) sin ——— 
ls% 


about (a) z = 0, (b) z= 1 and (c) z=, indicating 
the range of validity in each case. (Do not find terms 
explicitly; indicate only the form of the principal 
part.) 


Find the real and imaginary parts of the functions 
(a) e’sinhz (b) cos 2z 


(oe (a) ane 


Determine whether the following mappings are 


conformal, and, if not, find the non-conformal points: 


ue 


(a) w 


(b) w=22°+327+6(1 -j)z+1 


(c) w= 642+ 4 
Zz 


Consider the mapping w=cosz. Determine the points 
where the mapping is not conformal. By finding the 
images in the w plane of the lines x = constant and 
y = constant in the z plane (z = x + jy), draw the 
mapping similarly to Figures 4.14 and 4.18. 


Determine the location of and classify the 
singularities of the following functions: 


sin z ‘ 1 
(a) 2 (b) ey 
(c) 2t1  @ sechz 
Pa 


(e) sinhz (f) sin( 4) (g) 2? 


Find the residues of the following functions at the 
points indicated: 


Zl 


22 


23 


24 


(a) — (¢=-1) (b) “= (=n 
(1 +z) 2z-T 

Os c= @ (¢=-8) 
z-™ (z ae 


Find the poles and zeros, and determine all the 
residues, of the rational function 


f@ = (z - 1)(2° +3z+5) 


2(z' +1) 
Determine the residue of the rational function 


z’+6z-30z 
(z-1-j)° 


Evaluate the following contour integrals along 
the circular paths indicated: 


(a) fk 
ee PIE 


or) ( +1)(2 3 ar, 
(z° +9)(z° +4) 


b ote ps 
(c) where 

ee (Gil) Cis zl =2 
d 

se + ation GaN +i)’ 


where 


where C is |z| = 2 


where C is |z| = 


(i) Cis |e] = 

(Gil) Cig |= fil 
(e) ; ne ea 
@ (se IN ar Zar 1) 


(f) ; ote dz , where 
eAe=2) (@=2) 


, where C is |z—j| =} 


(i) Cis |z|=1 
(ii) C is |z| =3 


Using a suitable contour integral, evaluate the 
following real integrals: 


2 
a8 (be 


——— 
=e, (6h ae DY (GE ae Bae se 2) 


= 2) 
im | =e 
@ 2 +16 


* cos 20 dé 
of'2 


2n 
sin @ dO 
(©) [ 5+4cos 0 


4cos @ 


0 
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ol 


Introduction 


Laplace transform methods have a key role to play in the modern approach to the 
analysis and design of engineering systems. The stimulus for developing these methods 
was the pioneering work of the English electrical engineer Oliver Heaviside (1850- 
1925) in developing a method for the systematic solution of ordinary differential 
equations with constant coefficients. Heaviside was concerned with solving prac- 
tical problems, and his method was based mainly on intuition, lacking mathematical 
rigour: consequently it was frowned upon by theoreticians at the time. However, 
Heaviside himself was not concerned with rigorous proofs, and was satisfied that his 
method gave the correct results. Using his ideas, he was able to solve important 
practical problems that could not be dealt with using classical methods. This led to 
many new results in fields such as the propagation of currents and voltages along 
transmission lines. 

Because it worked in practice, Heaviside’s method was widely accepted by engineers. 
As its power for problem-solving became more and more apparent, the method attracted 
the attention of mathematicians, who set out to justify it. This provided the stimulus for 
rapid developments in many branches of mathematics including improper integrals, 
asymptotic series and transform theory. Research on the problem continued for many 
years before it was eventually recognized that an integral transform developed by the 
French mathematician Pierre Simon de Laplace (1749-1827) almost a century before 
provided a theoretical foundation for Heaviside’s work. It was also recognized that the 
use of this integral transform provided a more systematic alternative for investigating 
differential equations than the method proposed by Heaviside. It is this alternative 
approach that is the basis of the Laplace transform method. 

We have already come across instances where a mathematical transformation has 
been used to simplify the solution of a problem. For example, the logarithm is used to 
simplify multiplication and division problems. To multiply or divide two numbers, we 
transform them into their logarithms, add or subtract these, and then perform the 
inverse transformation (that is, the antilogarithm) to obtain the product or quotient of 
the original numbers. The purpose of using a transformation is to create a new domain 
in which it is easier to handle the problem being investigated. Once results have been 
obtained in the new domain, they can be inverse-transformed to give the desired results 
in the original domain. 

The Laplace transform is an example of a class called integral transforms, and it 
takes a function f(t) of one variable ¢ (which we shall refer to as time) into a function 
F(s) of another variable s (the complex frequency). Another integral transform widely 
used by engineers is the Fourier transform, which is dealt with in Chapter 8. The 
attraction of the Laplace transform is that it transforms differential equations in the t 
(time) domain into algebraic equations in the s (frequency) domain. Solving differ- 
ential equations in the f domain therefore reduces to solving algebraic equations 
in the s domain. Having done the latter for the desired unknowns, their values as 
functions of time may be found by taking inverse transforms. Another advantage of 
using the Laplace transform for solving differential equations is that initial conditions 
play an essential role in the transformation process, so they are automatically 
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Figure 5.1 Schematic 
representation of a 
system. 
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u(t) x(f) 
SYSTEM 
Input or Output or 


excitation response 


incorporated into the solution. This constrasts with the classical approach con- 
sidered in Chapter 10 of the companion text Modern Engineering Mathematics, 
where the initial conditions are only introduced when the unknown constants of 
integration are determined. The Laplace transform is therefore an ideal tool for solving 
initial-value problems such as those occurring in the investigation of electrical circuits 
and mechanical vibrations. 

The Laplace transform finds particular application in the field of signals and linear 
systems analysis. A distinguishing feature of a system is that when it is subjected to 
an excitation (input), it produces a response (output). When the input u(t) and output 
x(t) are functions of a single variable ¢, representing time, it is normal to refer to them 
as signals. Schematically, a system may be represented as in Figure 5.1. The problem 
facing the engineer is that of determining the system output x(t) when it is subjected to 
an input u(t) applied at some instant of time, which we can take to be t= 0. The relation- 
ship between output and input is determined by the laws governing the behaviour of 
the system. If the system is linear and time-invariant then the output is related to the 
input by a linear differential equation with constant coefficients, and we have a standard 
initial-value problem, which is amenable to solution using the Laplace transform. 

While many of the problems considered in this chapter can be solved by the classical 
approach, the Laplace transform leads to a more unified approach and provides the 
engineer with greater insight into system behaviour. In practice, the input signal u(f) 
may be a discontinuous or periodic function, or even a pulse, and in such cases the 
use of the Laplace transform has distinct advantages over the classical approach. Also, 
more often than not, an engineer is interested not only in system analysis but also in 
system synthesis or design. Consequently, an engineer’s objective in studying a sys- 
tem’s response to specific inputs is frequently to learn more about the system with a 
view to improving or controlling it so that it satisfies certain specifications. It is in this 
area that the use of the Laplace transform is attractive, since by considering the system 
response to particular inputs, such as a sinusoid, it provides the engineer with powerful 
graphical methods for system design that are relatively easy to apply and widely used 
in practice. 

In modelling the system by a differential equation, it has been assumed that both 
the input and output signals can vary at any instant of time; that is, they are functions 
of a continuous time variable (note that this does not mean that the signals themselves 
have to be continuous functions of time). Such systems are called continuous-time 
systems, and it is for investigating these that the Laplace transform is best suited. 
With the introduction of computer control into system design, signals associated with 
a system may only change at discrete instants of time. In such cases the system is said 
to be a discrete-time system, and is modelled by a difference equation rather than a 
differential equation. Such systems are dealt with using the z transform considered in 
Chapter 6. 
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The Laplace transform 


5.2.1 Definition and notation 


We define the Laplace transform of a function f(f) by the expression 
LY f(t)} = | enf(t) dt (5.1) 
0 
where s is a complex variable and e“ is called the kernel of the transformation. 
It is usual to represent the Laplace transform of a function by the corresponding 


capital letter, so that we write 


co 


LA fit)} = F(s) = | e"if(t) dt (5.2) 


0 


An alternative notation in common use is to denote ¥{ f(t)! by f(s) or simply /. 
Before proceeding, there are a few observations relating to the definition (5.2) worthy 
of comment. 


(a) The symbol £ denotes the Laplace transform operator; when it operates on a 
function /(f), it transforms it into a function F(s) of the complex variable s. We 
say the operator transforms the function f(4) in the ¢ domain (usually called the 
time domain) into the function F(s) in the s domain (usually called the complex 
frequency domain, or simply the frequency domain). This relationship is 
depicted graphically in Figure 5.2, and it is usual to refer to f(f) and F(s) as a 
Laplace transform pair, written as { (1), F(s)}. 


Figure 5.2 Lt} 
The Laplace transform 
operator. 
t domain s domain 
(time domain) (frequency domain) 


(b) Because the upper limit in the integral is infinite, the domain of integration is 
infinite. Thus the integral is an example of an improper integral, as introduced 
in Section 9.2 of Modern Engineering Mathematics; that is, 


| e" f(t) de = lim | e“f(t) dt 


This immediately raises the question of whether or not the integral converges, an 
issue we shall consider in Section 5.2.3. 


(c) Because the lower limit in the integral is zero, it follows that when taking the 
Laplace transform, the behaviour of f(t) for negative values of f is ignored or 
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Graph of f(t) and 

its causal equivalent 
function. 


(d) 
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suppressed. This means that F(s) contains information on the behaviour of /(f) 
only for ¢ = 0, so that the Laplace transform is not a suitable tool for investigating 
problems in which values of f(f) for ¢ < 0 are relevant. In most engineering applica- 
tions this does not cause any problems, since we are then concerned with physical 
systems for which the functions we are dealing with vary with time ¢. An attribute 
of physical realizable systems is that they are non-anticipatory in the sense 
that there is no output (or response) until an input (or excitation) is applied. 
Because of this causal relationship between the input and output, we define a 
function f(t) to be causal if f(f) = 0 (tf < 0). In general, however, unless the 
domain is clearly specified, a function /(f) is normally intepreted as being defined 
for all real values, both positive and negative, of t. Making use of the Heaviside 
unit step function H(t) (see also Section 5.5.1), where 


wn =|° (t <0) 
1 (t 20) 
we have 
0 (t<0) 
t)H(t) = 
ee i (t= 0) 


Thus the effect of multiplying /(‘) by H(A) is to convert it into a causal function. 
Graphically, the relationship between /(f) and f()H(A) is as shown in Figure 5.3. 


fd) f()H(Q) 


It follows that the corresponding Laplace transform F(s) contains full 
information on the behaviour of f(HA(f). Consequently, strictly speaking one 
should refer to { f(A(A, F(s)} rather than { f(), F(s)} as being a Laplace trans- 
form pair. However, it is common practice to drop the H(t) and assume that we 
are dealing with causal functions. 


If the behaviour of f() for t < 0 is of interest then we need to use the alternative 
two-sided or bilateral Laplace transform of the function /(‘), defined by 


Lat fi} = | e"'f(t) dt (5.3) 


The Laplace transform defined by (5.2), with lower limit zero, is sometimes 
referred to as the one-sided or unilateral Laplace transform of the function /(?). 
In this chapter we shall concern ourselves only with the latter transform, and refer 
to it simply as the Laplace transform of the function f(t). Note that when f(¢) is a 
causal function, 


Lai lO} = ALO} 
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(e) 


Another issue concerning the lower limit of zero is the interpretation of f(0) when 
f(é has a peculiarity at the origin. The question then arises as to whether or not 
we should include the peculiarity and take the lower limit as 0" or exclude it and 
take the lower limit as 0* (as conventional 0- and 0* denote values of t just to the 
left and right of the origin respectively). Provided we are consistent, we can take 
either, both interpretations being adopted in practice. In order to accommodate 
any peculiarities that may occur at ¢ = 0, such as an impulse applied at t= 0, we 
take 0" as the lower limit and interpret (5.2) as 


co 


LI fl} = F(s) = | e"if(t) dt (5.4) 


0- 


We shall return to this issue when considering the impulse response in Section 5.5.8. 


5.2.2 Transforms of simple functions 


Example 5.1 


Solution 


In this section we obtain the Laplace transformations of some simple functions. 


Determine the Laplace transform of the function 
f(D =e 


where c is a constant. 


Using the definition (5.2), 


be £ 
L£(c) -| e''cdt= im | ec dt 
To 0 
0 0 


T 
= lim E e = £ (: - lim e7) 
T00 S Ff S T00 


Taking s = 0+ j@, where o and @ are real, 


. -sT ~(6+j0)T. 
lim e*” = lim(e 
T30 


To 


) = lim e°’(cos @T + j sin oT) 
T- 00 


A finite limit exists provided that o= Re(s) > 0, when the limit is zero. Thus, provided 


that Re(s) > 0, the Laplace transform is 


Lc) = <. Re(s) > 0 


so that 
S(t) =e 
‘ Re(s) > 0 (5.5) 
F(s) = = 


constitute an example of a Laplace transform pair. 


www.2Ofile.org 


Example 5.2 


Solution 


Example 5.3 


Solution 
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Determine the Laplace transform of the ramp function 


fO=t 
From the definition (5.2), 


To 


i T 
Lith =| e''tdt= im | et dt 
0 0 


T-00 2 S T0 S T00 Ss 


oy Te" ee! 
= lim |_e*_€ = —-lim - lim 

2, 
s 


0 


Following the same procedure as in Example 5.1, limits exist provided that 
Re(s) > 0, when 


—sT —sT 


lim 
T00 Ss To Ss 


Thus, provided that Re(s) > 0, 
1 
Lit} = 
Ss 


giving us the Laplace transform pair 


Re(s) > 0 (5.6) 


Determine the Laplace transform of the one-sided exponential function 


f= e* 


The definition (5.2) gives 


To% 


oo T 
Len} = | ee" dt = lim | ee de 
0 


= Hee! [eeyr = 1 (1 ae een) 


Too §—k 0 s-k T00 


Writing s = 0+ j@, where o and @ are real, we have 


lim @ °°" = lime ""e 


T0 To% 


joT 


If k is real, then, provided that o = Re(s) > &, the limit exists, and is zero. If k is 
complex, say k= a+ jb, then the limit will also exist, and be zero, provided that o > a 
(that is, Re(s) > Re(x)). Under these conditions, we then have 


Le} = — 


1 
s-k 
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giving us the Laplace transform pair 


eos k (5.7) 
ll Re(s) > Re(k) : 
Sr a 


Example 5.4 _ Determine the Laplace transforms of the sine and cosine functions 
f() = sin at, g(t) = cos at 


where a is a real constant. 


Solution Since 
e = cos at + j sin at 
we may write 
f(@ = sin at = Im e!” 
g(t) = cos at = Re e!” 


Using this formulation, the required transforms may be obtained from the result 


Lie} =——, Re(s) > Re(k) 


1 
s-k 


of Example 5.3. 
Taking & = ja in this result gives 
F{e"y=+—, Re(s) > 0 
sS- ja 


or 


ely = S412 Res) > 0 
s +a 


Thus, equating real and imaginary parts and assuming s is real, 


L{sin at} = Im L{e} = —4 
S +a 


{cos at} = Re L{e”} = — 
sta 


These results also hold when s is complex, giving us the Laplace transform pairs 


L{sinat}=—*—, Re(s) > 0 (5.8) 
Ss ta 

L{cosat}=——, Re(s) > 0 (5.9) 
sta 
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&) In MATLAB, using the Symbolic Toolbox, the Laplace transform of a function f(t) 
is obtained by entering the commands 


Summ s € 
laplace(£(t)) 


with the purpose of the first command being that of setting up s and t as symbolic 
variables. 

To search for a simpler form of the symbolic answer enter the command 
simple (ans). Sometimes repeated use of this command may be necessary. To display 
the answer in a format that resembles typeset mathematics, use is made of the pretty 
command. Use of such commands will be illustrated later in some of the examples. 

If the function f(4) includes a parameter then this must be declared as a symbolic 
term at the outset. For example, the sequence of commands 


Svs S i el 
laplace(sin(a*t) ) 
gives, as required, 
ans=a/(S°2+a>2) 
as the Laplace transform of sin(a7). 
Use of MAPLE is almost identical to the MATLAB Symbolic Math Toolbox 
except for minor semantic differences. However, when using MAPLE the integral 


transform package must be invoked using inttrans and the variables ¢ and s must 
be specified explicitly. For instance the commands 


with(inttrans) : 
laplace(sin(a*t),t,s); 


return the transform as 


a 


S sa 


5.2.3 Existence of the Laplace transform 


Clearly, from the definition (5.2), the Laplace transform of a function /(f) exists if and 
only if the improper integral in the definition converges for at least some values of s. 
The examples of Section 5.2.2 suggest that this relates to the boundedness of the func- 
tion, with the factor e™ in the transform integral acting like a convergence factor in 
that the allowed values of Re(s) are those for which the integral converges. In order 
to be able to state sufficient conditions on f(t) for the existence of L{ f(t)}, we first 


introduce the definition of a function of exponential order. 


Definition 5.1 


A function f(4) is said to be of exponential order as t > if there exists a real 
number o and positive constants M and 7 such that 


If < Me* 


for all t > T. 
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What this definition tells us is that a function f(‘) is of exponential order if it does not 
grow faster than some exponential function of the form M e%’. Fortunately most functions 
of practical significance satisfy this requirement, and are therefore of exponential order. 
There are, however, functions that are not of exponential order, an example being e”, 
since this grows more rapidly than M e® as t > oo whatever the values of M and o. 


Example 5.5 — The function f(f) = e* is of exponential order, with o = 3. 


Example 5.6 — Show that the function f(#) = fF (t = 0) is of exponential order. 


Solution Since 
ea lore FO + FOP Pas 

it follows that for any a > 0 
6 


"3 
a 


r < ee 


so that ¢° is of exponential order, with o > 0. 


It follows from Examples 5.5 and 5.6 that the choice of o in Definition 5.1 is not 
unique for a particular function. For this reason, we define the greatest lower bound o, 
of the set of possible values of o to be the abscissa of convergence of f(¢). Thus, in the 
case of the function f(#) = e*, o, = 3, while in the case of the function /(/) = t°, o, = 0. 

Returning to the definition of the Laplace transform given by (5.2), it follows that 
if f(t) is a continuous function and is also of exponential order with abscissa of 
convergence O,, so that 


[f()|< Me", o>o, 


then, taking T= 0 in Definition 5.1, 


< | | ae 
0 


Writing s = 0 + j@, where o and ware real, since |e’| = 1, we have 


| f(2)| de 


IF(s)| = | | e"f(1) di 


le™| = le jes" = je | = ew 


so that 


co 


|F(s)| < | e” | f(n)| a= | eed. o> o. 
0 
=M 


0 
-(O-O4)t 
| e ° dt 
0 
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Figure 5.4 

Region of convergence 
for L{ f(t)}; 0, is 

the abscissa of 
convergence for /(7). 


Theorem 5.1 


5.2.4 
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This last integral is finite whenever o = Re(s) > oy. Since o, can be chosen arbitrarily 
such that o, > o, we conclude that F(s) exists for o > o,. Thus a continuous function 


f(t) of exponential order, with abscissa of convergence o,, has a Laplace transform 


Lf} = Fs), Re(s) > ©, 


where the region of convergence is as shown in Figure 5.4. 


Im(s) 


(a) ge>0 (b) o.<0 


In fact, the requirement that f(t) be continuous is not essential, and may be relaxed 
to f(t) being piecewise-continuous, as defined in Section 8.8.1 of Modern Engineering 
Mathematics; that is, f(t) must have only a finite number of finite discontinuities, being 
elsewhere continuous and bounded. 

We conclude this section by stating a theorem that ensures the existence of a Laplace 
transform. 


Existence of Laplace transform 


If the causal function /(f) is piecewise-continuous on [0, ce] and is of exponential order, 
with abscissa of convergence o,, then its Laplace transform exists, with region of con- 
vergence Re(s) > o, in the s domain; that is, 


o 


Lf f(t)} = F(s) -| eft) dt, Re(s) > o, 


0 


end of theorem 


The conditions of this theorem are sufficient for ensuring the existence of the Laplace 
transform of a function. They do not, however, constitute necessary conditions for 
the existence of such a transform, and it does not follow that if the conditions are 
violated then a transform does not exist. In fact, the conditions are more restrictive than 
necessary, since there exist functions with infinite discontinuities that possess Laplace 
transforms. 


Properties of the Laplace transform 


In this section we consider some of the properties of the Laplace transform that will 
enable us to find further transform pairs { f(t), F(s)} without having to compute them 
directly using the definition. Further properties will be developed in later sections when 
the need arises. 
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Example 5.7 


Solution 


Property 5.1: The linearity property 


A fundamental property of the Laplace transform is its linearity, which may be stated 
as follows: 


If f(t) and g(¢) are functions having Laplace transforms and if a and f are any 
constants then 


Ll af(d) + Pg} = OL f(D} + PAO} 


As a consequence of this property, we say that the Laplace transform operator Y is 
a linear operator. A proof of the property follows readily from the definition (5.2), 
since 


co 


Lh af(t) +Bg(t)} = | [afit) + Bg(t)]e™ de 


0 


=| anne” d+ | Bg(t)e" dt 


a faye" dr+ 6| g(t)e" de 
= at} f(t} + BL gd} 


Regarding the region of convergence, if f(¢) and g(t) have abscissae of convergence 0; 
and o, respectively, and 0, > 0;, 0) > O,, then 


ot o,t 
lfi)| < Me’, Ig(t)| < M,e° 
It follows that 


lft) + Be()| < lol || + [Bl g@1 S la] M, e+ |B 1M, e%" 
< (|a|M, + |B|M,) e” 
where O = max(0), 0), so that the abscissa of convergence of the linear sum 
of(t) + Bg(t) is less than or equal to the maximum of those for f(t) and g(t). 


This linearity property may clearly be extended to a linear combination of any finite 
number of functions. 


Determine F{31 + 2e*. 


Using the results given in (5.6) and (5.7), 


ey =4, Re(s) > 0 
AY 


Le} = at Rea Ss 
= 
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so, by the linearity property, 
L{3t + 2e*} = 3L{t} + 2F{e*} 


=3424, Re(s) > max{0, 3} 
Ss gs-3 

=3424, Re(s) > 3 

Ss g-3 


2) The answer can be checked using the commands 


MATLAB MAPLE 
syms s t with(inttrans) : 
llajllace (Sie 4 Qvexge(S ic) 7 leiolleca(se 4 2rerio( sie) 1, Ss) 4 


pretty (ans) 


which return 


— 
s= 3 


ee 
3 S=3 


aie 


Q |w 
w 


Example 5.8 = Determine {5 — 3+ 4sin 2r— 6e%}. 


Solution Using the results given in (5.5)—(5.8), 


Pisue 2, Re(s)>0 F{rp=4, Re(s) > 0 
iS: 


F{sin2t}= 2+, Re(sy)>0 Ffe"p=+, Res) > 4 
s +4 


s-4 


so, by the linearity property, 
SLA5 —3t+ 4sin2t— 6c} = L{5} -3L{t} + 4£{sin 27 — 6L{e*} 
_5_3 8-6 


= ot , Re(s) > max{0, 4} 
sos gs +4 g- 
=~2-3,8 - 6 Res) >4 
S gs gs +4 g- 
2) Again this answer can be checked using the commands 
Syvins Ss ic 
apiece (5 = 34 — Akcim(2*t)) = oriexp(4*t))) 


in MATLAB, or the commands 


with(inttrans) : 
lajolaee(S = Bic + Aesim(2 si) = 6 sexqo(4ie) , ic, Ss) es 
in MAPLE. 
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Theorem 5.2 


Proof 


Example 5.9 


Solution 


The first shift property is another property that enables us to add more combinations 
to our repertoire of Laplace transform pairs. As with the linearity property, it will prove 
to be of considerable importance in our later discussions particularly when considering 
the inversion of Laplace transforms. 


Property 5.2: The first shift property 


The property is contained in the following theorem, commonly referred to as the first 
shift theorem or sometimes as the exponential modulation theorem. 


The first shift theorem 


If f( is a function having Laplace transform F(s), with Re(s) > o,, then the function 
e“f(t) also has a Laplace transform, given by 


LF{ef(t)| = F(s—a), Re(s) > o, + Re(a) 


A proof of the theorem follows directly from the definition of the Laplace transform, since 


co co 


e"f(t)e" dt = | fpeo de 


0 


LLY} = | 


0 


Then, since 


co 


Li f(D} = F(s) =| Ane" dt, Re(s) > o, 


0 


we see that the last integral above is in structure exactly the Laplace transform of f(A) 
itself, except that s — a takes the place of s, so that 


Lite" f(t)} = F(s—a), Re(s—a)>o, 
or 
Life" f(t)} = F(s—a), Re(s) > o, + Re(a) 


end of theorem 


An alternative way of expressing the result of Theorem 5.2, which may be found 
more convenient in application, is 


Le f(O} = [LFOH sea = FO) 5-0 


In other words, the theorem says that the Laplace transform of e“ times a function f(t) 
is equal to the Laplace transform of f(t) itself, with s replaced by s — a. 


Determine L{te’?. 


From the result given in (5.6), 


Lt} = F(s)=4, Re(s) > 0 
iS 
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so, by the first shift theorem, 
Lt 7} = F(s+2)=[Fhou2, Re(s) > 0-2 


that is, 


Ce Re(s) > -2 
(s+ 


2)" 


=) This is readily dealt with using MATLAB or MAPLE. The commands 


MATLAB MAPLE 
syms st with(inttrans) : 
lajllace (ie sesqs) (=2°ie)) ) lajlace (eexgo (=2°1e)) 1b, S) 8 


pretty (ans) 
return the transform as 


te 
(ga 2)" 


Example 5.10 _—_— Determine Y{e™ sin 27}. 


Solution From the result (5.8), 


2 


s° +4” 


L{ sin 2t} = F(s) = Re(s) > 0 


so, by the first shift theorem, 
L{e™ sin 2t = F(s + 3) = [F(S)]ys3, Re(s) > 0-3 
that is, 


y) _ ) 


Lhe sin 2 = 
(s+3)+4 5° +65+13 


, Re(s) > -3 


2) In MATLAB the commands: 


Swamls, Sc 
laplace (exp (-3*t) *sin(2*t) ) 


return 
ais = 2B/((S + 3)°2 4 4) 


Entering the further commands 


simple(ans) ; 
pretty (ans) 
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Theorem 5.3 


returns 


a 


2/(s* + 6s 4+ 13) 


as an alternative form of the answer. Note that the last two commands could be 
replaced by the single command pretty (simple(ans) ). 
In MAPLE the commands 


with(inttrans) : 
lgiollece (Exga (=3'"E) “Sim (2) ES) F 


return the answer 


iL 


2 2 
(s13)-44 


There is no simple command in MAPLE. 


The function e’ sin 2t in Example 5.10 is a member of a general class of func- 
tions called damped sinusoids. These play an important role in the study of engi- 
neering systems, particularly in the analysis of vibrations. For this reason, we add 
the following two general members of the class to our standard library of Laplace 
transform pairs: 


a 


L{E"sin at} = ———,_ Re(s) > -k (5.10) 
(s+k) +a 

Le" cos at} = —St# —, Re(s) > -k (5.11) 
(st+k) +a 


where in both cases & and a are real constants. 


Property 5.3: Derivative-of-transform property 


This property relates operations in the time domain to those in the transformed s 
domain, but initially we shall simply look upon it as a method of increasing our 
repertoire of Laplace transform pairs. The property is also sometimes referred to as the 
multiplication-by-¢ property. A statement of the property is contained in the following 
theorem. 


Derivative of transform 


If f( is a function having Laplace transform 
F(s)=LAFO}, Re(s) > o, 


then the functions ¢” f(t) (n = 1, 2,...) also have Laplace transforms, given by 


Lt fly} = ae Re(s) > 0 
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Proof By definition, 


cs 


Li flt)} = F(s) = | ea" f(t) de 


0 


so that 
d'F(s)_ | Ay dt 
ds” ds” P 


Owing to the convergence properties of the improper integral involved, we can inter- 
change the operations of differentiation and integration and differentiate with respect to 
s under the integral sign. Thus 


a | F tepid 
ds 9 OS 


which, on carrying out the repeated differentiation, gives 


rad = (1) | Si" f(t) dt =(I'Lr" fF}, Re(s) > o, 


the region of convergence remaining unchanged. 


end of theorem 


In other words, Theorem 5.3 says that differentiating the transform of a function 
with respect to s is equivalent to multiplying the function itself by —t. As with the pre- 
vious properties, we can now use this result to add to our list of Laplace transform pairs. 


Example 5.11 Determine Y{t sin 37}. 


Solution Using the result (5.8), 
3 


ss +9" 


L{ sin 3t} = F(s) = Re(s) > 0 


so, by the derivative theorem, 


{1 sin 3t} = -AOD - 8 Re(s) > 0 
ds = (s° +9) 


2) In MATLAB the commands 


syms s t 
laplace(t*sin(3*t) ) 


return 
ams = i/(S*2 # 9) sam (2*eieain (3/5) ) 
Applying the further command 


simple(ans) 
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returns 
dims = G/(S°2 +¢ O)/s/(l += 9/82) 

Repeating the simple command 
simple(ans) 

returns the answer in the more desirable form 
ams = OFs/(S°2 + O)*2 

In MAPLE the commands 


with(intrans) : 
Jejollece (ic ealia (Se) , 12,8) ¢ 


return the transform as 


; i 
sin(2 arc tan(3 *)) 


s +9 


and there appears to be no command to simplify this. 


Example 5.12 Determine L{t’ e’}. 
Solution From the result (5.7), 
Pid} = Fs) = ome Re(s) > 1 
ee 


so, by the derivative theorem, 


2a —/_1)\2 PF(s 7 2a ih, 
Lire Cina cy S (4) 


ds*\s- 1 
d 1 
=(-l)— 
Far 


Re(s) > 1 


3? 


~ (9-1) 


Note that the result is easier to deduce using the first shift theorem. 


(2) Using MATLAB or MAPLE confirm that the answer may be checked using the 
following commands: 


MATLAB MAPLE 
syms st with(inttrans) : 
laplace(t*2*exp(t) ) eipllace (ie 2 eee) (ie) , ic, S) 5 
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Solution 


5.2.5 


Figure 5.5 

(a) Table of Laplace 
transform pairs; 

(b) some properties of 
the Laplace transform. 
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Determine S{t"}, where n is a positive integer. 


Using the result (5.5), 
#iy=4, Rew) = 0 
s 
so, by the derivative theorem, 


Lt} = cay Z(2) =2L. Re(s) > 0 


ds” Ss gt! $ 


Table of Laplace transforms 


It is appropriate at this stage to draw together the results proved to date for easy access. 
This is done in the form of two short tables. Figure 5.5(a) lists some Laplace transform 
pairs and Figure 5.5(b) lists the properties already considered. 


(a) . 

Sd) Li f(t)} = Fs) Region of convergence 

c, c a constant 7 Re(s) > 0 
s 

t i Re(s) > 0 
s 

| 

t", na positive integer a Re(s) > 0 

e“, ka constant a Re(s) > Re(k) 
<i 

sin at, a a real constant 5 a = Re(s) > 0 
Sta 

cos at, a a real constant = s ; Re(s) > 0 
sta 

e“ sin at, k and a real constants a 5 Re(s) > -k 
(stk) +a 

e“ cos at, k and a real constants s at 5 Re(s) > -k 
(stk) +a 

(b) 
Li f)} =F(s), Re(s) > o, and L{g(1)}=G(s), Re(s) > op 
Linearity: SL} af(t) + Be(t)} = aF(s) + BG(s), Re(s) > max(o,, o;) 
First shift theorem: Lie" f(t} = F(s— a), Re(s) > o, + Re(a) 


Derivative of transform: 


Lit" f(p)} = ee (n=1,2,...), Re(s) > 0, 
iS 
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5.2.6 Exercises 


1 Use the definition of the Laplace transform S 
to obtain the transforms of f(4) when f(A) is 


Using the results shown in Figure 5.5, obtain the 
Laplace transforms of the following functions, 


given by stating the region of convergence: 
(a) cosh 2t (b) ¢? (c) 3+¢ (d) te“ (a) 5—3t (b) 7¢? —2 sin3t 
stating the region of convergence in each case. (c) 3—2t+4cos2t (d) cosh 3¢ 

2 What are the abscissae of convergence for the A) Panes ae 
following functions? (g) 4te* (h) 2e~™ sin 2¢ 
(a) e* (b) e* (i) ?e* (j) 6t? — 34° + 4-2 
(c) sin 2t (d) sinh 3t (k) 2cos 3t+5sin3t (1) tcos2t 
(e) cosh 2t (f) «4 (m) f° sin 3¢ (n) t? —3cos4t 
(g) e% +r? (h) 3 cos2t— #7 (0) Pe*+e'cos2t+3 


(i) 3e%-2e7+ sin 2r 


(j) sinh 3¢ + sin 3¢ Check your answers using MATLAB or MAPLE. 


5.2.7. The inverse transform 


Figure 5.6 
The Laplace transform 
and its inverse. 


The symbol £'{F(s)} denotes a causal function f(f) whose Laplace transform is F(s); 
that is, 


if Lf f(t)} = F(s) 


This correspondence between the functions F(s) and f(t) is called the inverse 
Laplace transformation, f(t) being the inverse transform of F(s), and £' being 
referred to as the inverse Laplace transform operator. These relationships are depicted 
in Figure 5.6. 


then f(t) = L'{F(s)} 


ft 


ae 


As was pointed out in observation (c) of Section 5.2.1, the Laplace transform F(s) 
only determines the behaviour of f(t) for t = 0. Thus £'{F(s)} = f(d) only for t = 0. 
When writing £'{F(s)} = (0), it is assumed that ¢ = 0 so strictly speaking, we should 
write 


L {F(s)} = (OHO (5.12) 
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Example 5.15 


5.2.8 


EY 
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Since 


at 1 
oe aa aaa 


if follows that 


Since 


L{sin ot} = 


2 2 
Ss 


it follows that 


The linearity property for the Laplace transform (Property 5.1) states that if a@ and B 
are any constants then 


Ll af(t) + Bg(t)} = co L{ f(} + BL g())} = aF(s) + BG(s) 
It then follows from the above definition that 
L"{aF(s) + BG(s)} = af(t) + Bg(t) = aL '{F(s)} + BL" {G(s)} 


so that the inverse Laplace transform operator £" is also a linear operator. 


Evaluation of inverse transforms 


The most obvious way of finding the inverse transform of the function F(s) is to make 
use of a table of transforms such as that given in Figure 5.5. Sometimes it is possible 
to write down the inverse transform directly from the table, but more often than not 
it is first necessary to carry out some algebraic manipulation on F(s). In particular, we 
frequently need to determine the inverse transform of a rational function of the form 
P(s)/q(s), where p(s) and q(s) are polynomials in s. In such cases the procedure is first 
to resolve the function into partial fractions and then to use the table of transforms. 


Using MATLAB Symbolic Math Toolbox the commands 


Sve S 
ilaplace(F(s) ) 


return the inverse transform of F(s). The corresponding MAPLE commands are 


with(inttrans) : 
invlaplace(F(s),s,t); 
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Example 5.16 Find 


I 1 
od fersyre - 3] 


Solution — First 1/(s + 3)(s — 2) is resolved into partial fractions, giving 


al uy 
1 5 5 


(e300) ed 


Then, using the result Y'{1/(s + a)} = e“ together with the linearity property, we 


have 
gf! ee eee =-l1f 
(s+3)(s- 2) 2 


(2) Using MATLAB or MAPLE the commands 


MATLAB MAPLE 

syms st with(inttrans) : 
ileoleee(ily((s = Des = 2)) ie ilaolace(l/((s « 3)% 
pretty (ans) (Ss = 2) 588) 3 


return the anwers 


-zexp (-3t) + 1/5exp (2t) -ze0) & Bele 


Example 5.17 ‘Find 
Gf! stl 
ie +9) 


Solution Resolving (s + 1)/s*(s* + 9) into partial fractions gives 


stl] _ 
s°(s° +9) 


Using the results in Figure 5.5, together with the linearity property, we have 


ie ater = 5+5t-; cos 3f— 7 sin 3t 
s(s° +9) 
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2) Using MATLAB or MAPLE check that the answer can be verified using the 
following commands: 


MATLAB MAPLE 

syms s t with(inttrans) : 
ileolace((S # i) /(s*2e(62 & 8) ) ie invlaplace((s + 1)/ 
pretty (ans) (S29 (S°2 & ©) ) ,S, ie) 9 


5.2.9 Inversion using the first shift theorem 


In Theorem 5.2 we saw that if F(s) is the Laplace transform of /(#) then, for a scalar a, 
F(s — a) is the Laplace transform of e“f(t). This theorem normally causes little diffi- 
culty when used to obtain the Laplace transforms of functions, but it does frequently 
lead to problems when used to obtain inverse transforms. Expressed in the inverse form, 
the theorem becomes 


L'{F(s — a)} = ef(t) 
The notation 


L'{[F(s)] sat = 1/0] 


where F(s) = L{f(t)} and [F(s)],_,,.. denotes that s in F(s) is replaced by s — a, may 
make the relation clearer. 


Example 5.18 Find 


Solution 1 _ fi 
(s +2y s S—s+2 


and, since 1/s* = L{t1, the shift theorem gives 
z'| 1 ~ +e 
(s+2) 


=) Check the answer using MATLAB or MAPLE. 


www.2Ofile.org 


368 LAPLACE TRANSFORMS 


Example 5.19 Find 


A a — 
s+6s +13 


Solution 2 _ 2 ‘| 2 
S543 


+6s+13 (943/44 [52422 


and, since 2/(s* + 2”) = Y{sin 2¢}, the shift theorem gives 
. 2 = 
na ' 3S. DL rf =e ” sin 2t 
s +6s+ 13 


(Z) The MATLAB commands 


Syams; Ss) ic. 
ilaplace(2/(s*2 + 6*s + 13); 
pretty (simple(ans) ) 


return 
ans = -1/2i(exp((-3 + 21)t) - exp((-3 - 2i)t)) 
The MAPLE commands 


with(inttrans) : 
invlaplace(2/(s*2 + 6*s + 13),s,t); 
Salimoll si (3) 


return the same answer. 
To obtain the same format as provided in the solution further manipulation is 
required as follows: 


1/2i[—e*e™" + ee" ] = & ( (0° — €*")/(2i) ) = €* sin 2t 


Example 5.20 ‘Find 


1 oe 7 
£ 2 
oe 


Solution st]... st7 
S+2s+5 (s+1)4+4 
__(s+l)) 43 2 


(s+1 +4 (s+) +4 
s 2 
F + | F + 7 sostl 
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Solution 


EY 
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Since s/(s* + 2”) = L{cos 2t} and 2/(s* + 2”) = L{sin 21}, the shift theorem gives 


- +7 -t 2h 
x aes = @'cos 2¢+3e’ sin 2¢ 
s+2s+5 


Find 


oa 
(s+ 1)°(s° +4) 


Resolving 1/(s + 1)°(s* + 4) into partial fractions gives 
2 


| 3, § 1 2843 


(s+1)(° +4) s41 


ee eee 1 2 Ss 3 2 
~ S| 2 “25°22 502. v2 
stl iS Wee s+2 s +2 


Since 1/s* = L{t}, the shift theorem, together with the results in Figure 5.5, gives 


(stl? * 5744 


1 _ at S 


Check the answers to Examples 5.20 and 5.21 using MATLAB or MAPLE. 


5.2.10 Exercise 


Check your answers using MATLAB or MAPLE. 


4 Find £'{F(s)} when F(s) is given by (j) fe= 1G Die= 3) (k) (5 +3)(s? +2) 
1 s +5 
b —__ ss Te S s- 1 
(a) (s +3)(s +7) (») (s +1)(s - 3) @ (s-1)(s° +25 +2) o gy +29 +5 
- oe (d) a (n) 2 = 8 21 —- (0) 3s 
s(s +3) a: (s - 2)(s - 3)(s - 4) (s - 1)(s?- 4) 
=e @ @) 2 
s(s° +16) s +4s +5 s(s° +1)(s° +9) 
s+ 4s 25° +45 +9 
= rear Qi 
s°(s° +45 +8) (s- 1)(s +1) (s +2)(s° +38 +3) 
(i) s+7 ee 
gs” +25 +5 (s +1)(s +2)(s° +2s +10) 
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Solution of differential equations 


5.3.1 


We first consider the Laplace transforms of derivatives and integrals, and then apply 
these to the solution of differential equations. 


Transforms of derivatives 


If we are to use Laplace transform methods to solve differential equations, we need 
to find convenient expressions for the Laplace transforms of derivatives such as df/dt, 
f/dt* or, in general, d"f/dt". By definition, 


GEN | gw Ley, 
dt >» at 
Integrating by parts, we have 


co 


off =[e"fNlots | e'if(t)dt= —f(0) + sF(s) 


0 


that is, 


fh = sF(s) — f(0) (5.13) 


In taking the Laplace transform of a derivative we have assumed that f(t) is continuous 
at t = 0, so that f(0-) = f(0) = f(0*). In Section 5.5.8, when considering the impulse 
function, f(0-) # f(0*) and we have to revert to a more generalized calculus to resolve 
the problem. 

The advantage of using the Laplace transform when dealing with differential equations 
can readily be seen, since it enables us to replace the operation of differentiation in the 
time domain by a simple algebraic operation in the s domain. 

Note that to deduce the result (5.13), we have assumed that f(f) is continuous, with 
a piecewise-continuous derivative df/dt, for t = 0 and that it is also of exponential order 
as f > 0, 

Likewise, if both f(f) and df/dt are continuous on t = 0 and are of exponential order 
as t — 0, and d’f/dt* is piecewise-continuous for ¢ > 0, then 


ott} | e" Ey fe" ea anh gy = 4 sl 
dr 0 dr d 0 0 dt dt 1=0 dt 
which, on using (5.12), gives 
d'f| __| or : 
i) ff) ester) f(0)] 


leading to the result 


oat = s F(s) - sf(0) - El = sF(s) - sf(0) - f(0) (5.14) 
t 


t=0 
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Clearly, provided that f() and its derivatives satisfy the required conditions, this pro- 
cedure may be extended to obtain the Laplace transform of f(t) = d”f/dt" in the form 


LAS} = s"F(s) - 8" FO) - 8" FO) —...-f°%O) 
= s'F(s)- 870) (5.15) 


i=1 


a result that may be readily proved by induction. 
Again it is noted that in determining the Laplace transform of f(f) we have 
assumed that f"(¢) is continuous. 


Transforms of integrals 


In some applications the behaviour of a system may be represented by an integro- 
differential equation, which is an equation containing both derivatives and integrals 
of the unknown variable. For example, the current 7 in a series electrical circuit con- 
sisting of a resistance R, an inductance L and capacitance C, and subject to an applied 
voltage E, is given by 


d..i{. 7 
Lgeined| coar=e 


To solve such equations directly, it is convenient to be able to obtain the Laplace 
transform of integrals such as fj f(t) dt. 
Writing 


«=| Kt) de 


we have 


dg _ _ 
Ar BO 


Taking Laplace transforms, 


dg | _ 
| iE} = 10} 


which, on using (5.13), gives 
sG(s) = F(s) 


or 


££ g(t) } = G(s) + F(s) + LK} 


leading to the result 


{| fit) ar} 2 + Lf} = + F(s) (5.16) 
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Example 5.22 


5.4.5 


Obtain 


| (¢ + sin 27) a 


In this case f(f) = f° + sin 2t, giving 


F(s) = LEf()} = LEP} + Lfsin 24 


6 2 
=-+—7—-— 
ss +4 
so, by (5.16), 
#{| (2 +sin20 dr = Le) = $4 2 
0 S gs s(s +4) 


Ordinary differential equations 


Having obtained expressions for the Laplace transforms of derivatives, we are now in 
a position to use Laplace transform methods to solve ordinary linear differential equations 
with constant coefficients. To illustrate this, consider the general second-order linear 
differential equation 
2 
ao pHs ox =u(t) (t= 0) (5.17) 
dr dt 
subject to the initial conditions x(0) = xo, x(0) = vy) where as usual a dot denotes differ- 
entiation with respect to time, ¢. Such a differential equation may model the dynamics 
of some system for which the variable x(t) determines the response of the system to the 
forcing or excitation term u(t). The terms system input and system output are also 
frequently used for u(t) and x(t) respectively. Since the differential equation is linear 
and has constant coefficients, a system characterized by such a model is said to be a 
linear time-invariant system. 
Taking Laplace transforms of each term in (5.17) gives 


og) + oat | +cL£{x} = L{u(t)} 
dt dt 


which on using (5.13) and (5.14) leads to 
a[s*X(s) — sx(0) — x(0)] + b[sX(s) — x(0)] + cX(s) = U(s) 
Rearranging, and incorporating the given initial conditions, gives 
(as* + bs + c)X(s) = U(s) + (as + b)xy + avy 
so that 


U(s) + (as + b)x9 + avy 


X(s) = (5.18) 


as +bs+c 
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Equation (5.18) determines the Laplace transform X(s) of the response, from which, by 
taking the inverse transform, the desired time response x(t) may be obtained. 


Before considering specific examples, there are a few observations worth noting at 


this stage. 


(a) 


(b) 


(c) 


(d) 


As we have already noted in Section 5.3.1, a distinct advantage of using the 
Laplace transform is that it enables us to replace the operation of differentiation 
by an algebraic operation. Consequently, by taking the Laplace transform of each 
term in a differential equation, it is converted into an algebraic equation in the 
variable s. This may then be rearranged using algebraic rules to obtain an expres- 
sion for the Laplace transform of the response; the desired time response is then 
obtained by taking the inverse transform. 


The Laplace transform method yields the complete solution to the linear differ- 
ential equation, with the initial conditions automatically included. This contrasts 
with the classical approach, in which the general solution consists of two compo- 
nents, the complementary function and the particular integral, with the initial 
conditions determining the undetermined constants associated with the comple- 
mentary function. When the solution is expressed in the general form (5.18), upon 
inversion the term involving U(s) leads to a particular integral while that involv- 
ing x) and v) gives a complementary function. A useful side issue is that an 
explicit solution for the transient is obtained that reflects the initial conditions. 


The Laplace transform method is ideally suited for solving initial-value prob- 
lems; that is, linear differential equations in which all the initial conditions 
x(0), x(0), and so on, at time ¢ = 0 are specified. The method is less attractive for 
boundary-value problems, when the conditions on x(f) and its derivatives are not 
all specified at t = 0, but some are specified at other values of the independent 
variable. It is still possible, however, to use the Laplace transform method by 
assigning arbitrary constants to one or more of the initial conditions and then 
determining their values using the given boundary conditions. 


It should be noted that the denominator of the right-hand side of (5.18) is the left- 
hand side of (5.17) with the operator d/dt replaced by s. The denominator equated 
to zero also corresponds to the auxiliary equation or characteristic equation used 
in the classical approach. Given a specific initial-value problem, the process of 
obtaining a solution using Laplace transform methods is fairly straightforward, 
and is illustrated by Example 5.23. 


Solve the differential equation 


dix | 5 dx 


+6x=2e' (t=0) 
dr dt 


subject to the initial conditions x = 1 and dx/dt= 0 at t= 0. 


Taking Laplace transforms 


ei fel, seh 6 Lx} =2 LE} 
dr dt 
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leads to the transformed equation 


2 


[s2X(s) — sx(0) — X(0)] + 5[sX(s) — x(0)] + 6X(s) = ox 


which on rearrangement gives 
(s° + 5s + 6)X(s) = a + (s + 5)x(0) + x(0) 

Incorporating the given initial conditions x(0) = 1 and x(0) = 0 leads to 
(s? + 5s + 6)X(s) = +545 


That is, 


a a ee ss ee 
(st+1)(s+2)(s+3) (s+3)(s+2) 


Resolving the rational terms into partial fractions gives 


X(s) 


Hie el 
stl s+2 s+3 st+2 gs+4+3 


ecg ll et 
stl s+2 g+3 
Taking inverse transforms gives the desired solution 


xh=e’+e%-e*% (t= 0) 


In principle the procedure adopted in Example 5.23 for solving a second-order linear 
differential equation with constant coefficients is readily carried over to higher-order 
differential equations. A general nth-order linear differential equation may be written as 


n—l 
a, 224,024... 4a =u(2) (t = 0) (5.19) 
de’ dr”” 
where a,, d,;, -.-, @ are constants, with a, # 0. This may be written in the more 
concise form 
q(D)x(t) = u(t) (5.20) 


where D denotes the operator d/d¢ and q(D) is the polynomial 


n 


q(D) =} a,D’ 


r=0 
The objective is then to determine the response x(t) for a given forcing function u(f) 
subject to the given set of initial conditions 


ox) = [& =c, (r=0,1,...,n-1) 
t=0 


Taking Laplace transforms in (5.20) and proceeding as before leads to 
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X(s) = 2) 
q(s) 


where 


n-1 n 
p(s) = U(s) + o3 C, »y a 
r=0 i=rtl1 
Then, in principle, by taking the inverse transform, the desired response x(t) may be 
obtained as 


_ fee) 
re (ea 


For high-order differential equations the process of performing this inversion may 

prove to be rather tedious, and matrix methods may be used as indicated in Section 5.7. 
To conclude this section, further worked examples are developed in order to help 

consolidate understanding of this method for solving linear differential equations. 


Solve the differential equation 


2 
Gx 6H 40x = sine (t = 0) 
dr dt 


subject to the initial conditions x = 0 and dx/dt= 0 at t=0. 


Taking the Laplace transforms 


(fs " 6se| th POG = Orsini 


leads to the equation 


[s2X(s) — sx(0) — X(0)] + 6[sX(s) — x(0)] + 9X(s) = 


2 
So+ 
which on rearrangement gives 


(s? + 6s + 9)X(s) = 


1 + (s+ 6)x(0) +x(0) 

s+] 

Incorporating the given initial conditions x(0) = x(0) = 0 leads to 
1 


ee oe 
(s+ 1)(s+3) 


Resolving into partial fractions gives 


1 1 1 5 
X(s)=i——+4t +2———-i-*— 
ges Viggo gd 
that is, 
1 1 1 s 
X(s) =i—+14]= +2 = 3S 
YB 10 ri er ca 0 4 
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Taking inverse transforms, using the shift theorem, leads to the desired solution 


3. St 1 —3 t 20: 3 
x(t) = GE +7,te +Hsmt-scost (t2=0) 


(2) In MATLAB, using the Symbolic Math Toolbox, the command dsolve computes 
symbolic solutions to differential equations. The letter D denotes differentiation 
whilst the symbols D2, D3,..., DN denote the 2™', 3™,..., N” derivatives respec- 
tively. The dependent variable is that preceded by D whilst the default independent 
variable is t. The independent variable can be changed from t to another symbolic 
variable by including that variable as the last input variable. The initial conditions 
are specified by additional equations, such as Dx(0) = 6. If the initial conditions 
are not specified the solution will contain constants of integration such as C1 and C2. 

For the differential equation of Example 5.24 the MATLAB commands 


Swans; =< ic 
x = dsoliwa(D2. + 63D 4 9m = Siin(ic)’, (0) = ©, (O) = 
Oi") § 


pretty (simple (x) ) 
return the solution 
x = -3/50cos(t) + 2/25sin(t) + 3/50(1/exp(t)?) 
= 1/10 (ie/esao (ic) ”) 
It is left as an exercise to express 1/exp(t)° ase. 
In MAPLE the command dsolve is also used and the commands 
Ceres Caine Ge(ic) ce) = Omer (Ge(ic) 6) = See) = Siim(e) e 
clsolwe({ocde2, s2(0) = ©, DG) (0) = O8, Re) )e 
return the solution 


S See dey eae 
(Ce) = eg° a to0° Epcos (t) + agein(t) 


If the initial conditions were not specified then the command 
dsolve({ode2}, x(t)); 


returns the solution 


— pal-3t) (-3t) = 2h 2 oe 
Scie) =e Cl +e ee ee Epcos (t) + spin (t) 
In MAPLE it is also possible to specify solution by the Laplace method and the 
command 
clsolwe({ecde2, (0) = 0, Dis) (O) = Oh, ie), 


method = laplace) ; 


also returns the solution 
il 


x(t) = -Zcos (t) PI Asinit) + Bel (5t + 3) 


and, when initial conditions are not specified, the command 
dsolve({ode2},x(t), method = laplace) ; 


returns the solution 
ees ei: (te 
Sale) Epcos (t) + spsin(t) + 50° (SO ieiw)(ez)) (10) 


+ WSO % s(@)) + Sic + Sse) + 3) 
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Example 5.25 — Solve the differential equation 
Px dx, 17% 4 13x=1 G20) 
dé dr dt 
subject to the initial conditions x = dx/dt = 1 and d’x/dt? = 0 at t=0. 


Solution Taking Laplace transforms 
w(t she seit she irae 13L{x} = L{1} 
dr’ dr? dt 
leads to the equation 
sX(s) — s°x(0) — sx(0) — ¥(0) + 5[s*X(s) — sx(0) — x(0)] 


+ 17[sX(s) — x(0)] + 13.X(s) = : 
s 
which on rearrangement gives 
(s' + 5s? + 17s + 13)X(s) = = + (s° + Ss + 17)x(0) + (s + 5)x(0) + ¥(0) 


Incorporating the given initial conditions x(0) = x(0) = | and x(0) = 0 leads to 


3 2 
Xs) = s + os" +225+1 
s(s +5s° + 17s + 13) 


Clearly s + 1 is a factor of s* + 5s? + 17s + 13, and by algebraic division we have 


3 2. 
X(s) = s +6s ee 
s(s+1)(s° +45 +4 13) 


Resolving into partial fractions, 
1 8 1 


Ms) = 843-1 44s+7_ 8, $1 44(s +2) - 27(3) 


s stl “s+45+13 8 stl % (s+2)°+3° 
Taking inverse transforms, using the shift theorem, leads to the solution 


x(t) = $+ $e'- Le"(44 cos 34-27 sin3/) (¢> 0) 


=) Confirm that the answer may be checked using the commands 


syms x t 
x = Cholvwea( ws & Sets - IomRe + see & 1’, Ma (0) = i, 
DAs (0) = OF) pz 
pretty (simple (x) ) 
in MATLAB, or the commands 
o¢less = ¢chirt(e(te), ©Sa)) + Schutt (<(ic), ©S2) 
45 7 sGluirie (SCE) fe) + ASese(ie) = as 
6ls@live(toces!,<(0) = 1, DG<) (0) = 4, (DEED) Ee) (C) = OF, 


x(t) ,method = laplace); 
in MAPLE. 
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5.3.4 Simultaneous differential equations 


Example 5.26 


Solution 


In engineering we frequently encounter systems whose characteristics are modelled 
by a set of simultaneous linear differential equations with constant coefficients. The 
method of solution is essentially the same as that adopted in Section 5.3.3 for solving 
a single differential equation in one unknown. Taking Laplace transforms throughout, 
the system of simultaneous differential equations is transformed into a system of 
simultaneous algebraic equations, which are then solved for the transformed variables; 
inverse transforms then give the desired solutions. 


Solve for tf = 0 the simultaneous first-order differential equations 


dx .d et 

ata txt3y=e (5.21) 
dx , dy = 

aa gee (5.22) 


subject to the initial conditions x = 2 and y= 1 at t=0. 


Taking Laplace transforms in (5.21) and (5.22) gives 
sX(s) — x(0) + s¥(s) — y(0) + 5X(s) + 3 Y(s) = + 
2[sX(s) — x(0)] + s¥(s) — (0) + X(s) + ¥(s) = ; 


Rearranging and incorporating the given initial conditions x(0) = 2 and (0) = 1 
leads to 


_y,_1 _3s+4 
(s+ 5)4is) + G+ 3)Ms) =3+ 5 = re; (5.23) 
(2s + 1)X(s) + (s+ DMs) =5+ ; = acts (5.24) 


Hence, by taking Laplace transforms, the pair of simultaneous differential equations 
(5.21) and (5.22) in x(t) and y(t) has been transformed into a pair of simultaneous 
algebraic equations (5.23) and (5.24) in the transformed variables X(s) and Y(s). 
These algebraic equations may now be solved simultaneously for X(s) and Y(s) using 
standard algebraic techniques. 

Solving first for X(s) gives 


_ Os + 14e49 


BO rs Ole) 


Resolving into partial fractions, 


9 1 25 


Raye? 
(s) s g+2 s-1 
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which on inversion gives 
x(t) =-$- FE"+ 3 (t= 0) Ges) 
Likewise, solving for Y(s) gives 
3 2 
BS) = re a ea 
Resolving into partial fractions, 


15: 1 ul 25 


¥(s)=F4+—2 42-2 
s stl s+2 s-l 


which on inversion gives 


Lot, Ut 2s 


y= e+se+Fe'-2e (¢ = 0) 


Thus the solution to the given pair of simultaneous differential equations is 
x(t)=-2-He"+Be | 
t 
y(t)=B+ie'+tev-Be 


Note: When solving a pair of first-order simultaneous differential equations such as 
(5.21) and (5.22), an alternative approach to obtaining the value of y(t) having obtained 
x(t) is to use (5.21) and (5.22) directly. 

Eliminating dy/dt from (5.21) and (5.22) gives 


2y= e —4x-3+e" 
Substituting the solution obtained in (5.25) for x(t) gives 
2y=(Fe"%+2e@)-4(-2- e"+%)e-34+e" 
leading as before to the solution 


= -2 
y=Hele's¥e”-¥e 


A further alternative is to express (5.23) and (5.24) in matrix form and solve for X(s) 
and Y(s) using Gaussian elimination. 


In MATLAB the solution to the pair of simultaneous differential equations of 
Example 5.26 may be obtained using the commands 

syms xX y t 

rll = cliolinva( wpe & Dy 4 See 2 Biv = edo)”, 

Ue D pig wats IDE te ote ee yeh ahr. 

E\((0)) = 2pxr(O)) = 1%) 
which return 

se = SIN /eresqo( 26) + A25/sexqo (ie) 9/2 

Wy = =25/2-esgo(e)) + lil/Qreedo(-2ie) + 15/2 = I/2serqo (=) 
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These can then be expressed in typeset form using the commands pretty (x) and 
pretty (y). In MAPLE the commands 


ocele= DG) (ie) + Diy) Ce) & Sexe) + Beye) = Eso(—ic) - 
oceee= 2D (xe) (ie) + wii) Ce) + See) = sre) = Be 
) 


Clsolwa(tocel,ocde2, (0) = a,y(@) = 1h, tele) vic) }) 2 
return 
_ al ae) 25 g) 29) 2. Ll ay, IS Lite) 
fec(ie)) = 6 3° Ne) a See } 


In principle, the same procedure as used in Example 5.26 can be employed to solve a 
pair of higher-order simultaneous differential equations or a larger system of differen- 
tial equations involving more unknowns. However, the algebra involved can become 
quite complicated, and matrix methods, considered in Section 5.7, are usually preferred. 


5.3.5 Exercises 


Check your answers using MATLAB or MAPLE. 


5 — Using Laplace transform methods, solve for t = 0 (g) 
the following differential equations, subject to the 


specified initial conditions: 
dy -2¢ 

a) —+3x=e 

(a) a 
subject tox =2 att=0 


(b) 3% _ ay = sin 2 


dt 
subject to x= + atr=0 
(c) dix $2 p51 
dr’ dt 


subject to x = 0 and a9 at t=0 


(d) dy +22 ty =4 cos 2t 
t 


dr’ 

subject to y = 0 and a) at t=0 
(e) dx _ 49% 49, - 76" 

dt dt 

subject to x = 0 and a at t=0 
(f) dx 44% 4572362 

dt dt 

subject to x = 4 and a7 att=0 


(h) 


(i) 


(j) 


(k) 


(I) 
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) Pz 4dr 9, 256 sin ¢ 
de dt 
subject to x = | and a9 at t=0 


9) 


dy 9D 43, = 37 


tm 
subject to y = 0 and oy att=0 
Px 448 44 P em 
di? va 
subject to x =} and a9 att=0 
gz 12% 457 =1 

dt dr 
subject to x = 0 and or 9 at t=0 
Pr gc = +16x = 16 sin 4t 
dt dr 
subject to x = - and a att=0 


9 dy +122 +4y=e' 
dr dr 


subject to y = | and a at t=0 
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dx .d 


3 
(my P29 Fx_ Oyo no 4s (e) a ae 2x =3 sint +5 cost 
dé drs dt dr 
2 
eg and SF =0 at t=0 2S 4G y= sin r+ cos 1 
(n) 2 ae fy Ox 4 de +x = cos 3¢ subject tox = 0 and y=—I at r=0 
de dr d us 
: f) S4+24y=t 
subject tox=0, % =1 and $2 =1 te=0 ) diode 
dt dt 
de +4 dy +x=1 
6 Using Laplace transform methods, solve for t = 0 dt dt 


the following simultaneous differential equations 


: . be das, a subject tox = 1 andy=0 att=0 
subject to the given initial conditions: 


dx dy 
dx, dy e (g) 25% 43 2 47x = 14¢ +7 
gauge oy ee dt — dr 
@) dood 
de _ dy = (ay 
28 44M t4r-37y=0 53 Gq tax ty = l4r- 14 
subject tox = 0 and y=} atr=0 subject tox =y =0 at t=0 
(b) de Way y=5sint (h) ar, Gy = yay 
dt dt dr dt 
5 dk dy subject to x = 4, y = 2, dx/dt = 0 and dy/dt = 0 
ar +34 te y=e esi 
2 
subject tox = 0 and y=0 at t=0 (i) eee 


dr 
(c) de W495 ty=e° 


de * de 5 ox +16 42 y +6y =0 
ay : me dr 
45x 4+3y=5e~ 5 
dt subject tox= 1, y= 1, dx/dt=0 and dy/dt=0 


at t=0 


ydx_ dy _de_dy_ 


subject tox =—1 and y=4 att=0 


dx ..d (j) = 3y-9x 
2 ae 2x=€ d’? de d de 
de 4,dv_ 4 2 dx _ dy , de dy 5y - 7x 
di dt de de dt dr 
subject tox = 1 andy=1 att=0 subject to x = dx/dt= 1 and y= dy/dt=0 at t=0 


ae ee aadiacalicayeieriece electrical circuits and 


mechanical vibrations 


To illustrate the use of Laplace transforms, we consider here their application to the 
analysis of electrical circuits and vibrating mechanical systems. Since initial con- 
ditions are automatically taken into account in the transformation process, the Laplace 
transform is particularly attractive for examining the transient behaviour of such 
systems. 
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5.4.1 


Figure 5.7 
Constituent elements 
of an electrical circuit. 


Example 5.27 


Using the commands introduced in previous sections MATLAB or MAPLE can be 
used throughout this section to check answers obtained. 


Electrical circuits 


Passive electrical circuits are constructed of three basic elements: resistors (having 
resistance R, measured in ohms Q), capacitors (having capacitance C, measured in 
farads F) and inductors (having inductance L, measured in henries H), with the asso- 
ciated variables being current i(t) (measured in amperes A) and voltage v(t) (measured 
in volts V). The current flow in the circuit is related to the charge g(t) (measured in 
coulombs C) by the relationship 


= 
dt 


Conventionally, the basic elements are represented symbolically as in Figure 5.7. 


+ R + Gua-4 + L - 
o—l___}—_ o——f ——— 
i({)—» i(t)—> i(t)}—» 

(a) Resistor (b) Capacitor (c) Inductor 


The relationship between the flow of current i(t) and the voltage drops v(t) across 
these elements at time f¢ are 


voltage drop across resistor = Ri (Ohm’s law) 


voltage drop across capacitor = + ji C= é 


The interaction between the individual elements making up an electrical circuit is deter- 
mined by Kirchhoff’s laws: 


Law 1 


The algebraic sum of all the currents entering any junction (or node) of a circuit is zero. 


Law 2 


The algebraic sum of the voltage drops around any closed loop (or path) in a circuit is zero. 


Use of these laws leads to circuit equations, which may then be analysed using Laplace 
transform techniques. 


The LCR circuit of Figure 5.8 consists of a resistor R, a capacitor C and an inductor L 
connected in series together with a voltage source e(t). Prior to closing the switch at 
time t = 0, both the charge on the capacitor and the resulting current in the circuit are 
zero. Determine the charge q(t) on the capacitor and the resulting current i(f) in the 
circuit at time ¢ given that R= 160 Q, L=1H, C= 107% F and e(t) = 20 V. 
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Figure 5.8 
LCR circuit of 
Example 5.27. 


Solution Applying Kirchhoff’s second law to the circuit of Figure 5.8 gives 


ar%s lla 
nist ded [id= etn (5.26) 


or, using 7 = dg/dt, 
2 
£944 R%41 9-002) 
dt dt C 
Substituting the given values for L, R, C and e(t) gives 
fa, 160% + 10% = 20 
dt dt 


Taking Laplace transforms throughout leads to the equation 
(s* + 160s + 10*)Q(s) = [sq(0) + G(0)] + 160q(0) + 20 
S 
where Q(s) is the transform of g(f). We are given that g(0) = 0 and q(0) = i(0) = 0, so 
that this reduces to 


(s? + 160s + 105.Q(s) = 22 
S 
that is, 


20 


0 
s(s + 160s + 10°) 


Resolving into partial fractions gives 


1 


500 + 160 
0) =" == 
s 4160s + 10° 


,|L (s+80)+53(60)} , 1 fs + $x 60 
~ 500 Ft (s+ 80)? + (60) ~~ 500 ig 2 2 — 


Taking inverse transforms, making use of the shift theorem (Theorem 5.2), gives 


g(t) = 3, (1 — e* cos 60t — $e sin 602) 


The resulting current i(¢) in the circuit is then given by 


i(t) = ce = te” sin 607 


www.2Ofile.org 


384 LAPLACE TRANSFORMS 


Example 5.28 


Figure 5.9 
Parallel circuit of 
Example 5.28. 


Solution 


Note that we could have determined the current by taking Laplace transforms in (5.26). 
Substituting the given values for L, R, C and e(t) and using (5.26) leads to the trans- 
formed equation 


4 

160i(s) + s1(s) + 42 15) = 28 

s Ss 
that is, 
= 20 = _ 
Sys «CO 8) since (0) = 0) 
(s° + 80) + 60° 

which, on taking inverse transforms, gives as before 

i(t) =1e°" sin 604 


In the parallel network of Figure 5.9 there is no current flowing in either loop prior to 
closing the switch at time tf = 0. Deduce the currents 7,(¢) and 7,(t) flowing in the loops 
at time f. 


R, = 202 L,=0.5H 


e(t) = 200V ( ) 


Applying Kirchhoff’s first law to node X gives 
i=i,t+i 
Applying Kirchhoff’s second law to each of the two loops in turn gives 


RG, +i)4+L, AG + i) + Roi, = 200 


Ie = + Ryi,— Ryi, = 0 


Substituting the given values for the resistances and inductances gives 


a. o. 561, + 407, = 400 
4 (5.27) 
Gi 8 t 10! =0 


Taking Laplace transforms and incorporating the initial conditions i,(0) = 7,(0) = 0 
leads to the transformed equations 
_ 400 
(s + 56)/\(s) + (s + 40)L,(s) = —— (5.28) 
Ss 


-8/,(s) + (s + 10)L,(s) = 0 (5.29) 
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Hence 


higs 3200 _ 3200 
s(s° + 745+ 880) s(s+59.1)(s+ 14.9) 


Resolving into partial fractions gives 


3.64 | 1.22 _ 4.86 
Ss eran s+ 14.9 


L,(s) = 


which, on taking inverse transforms, leads to 
i,(t) = 3.64 + 1.22 el A 86 eo 4 
From (5.27), 


i,(t) = :(10i, + a) 


that is, 
I(t) = 4.55 — 7.49 ee" + 2.98 @ 14% 


Note that as tf — », the currents i,(f) and i,(t) approach the constant values 4.55 and 3.64 
A respectively. (Note that i(0) = 7,(0) + 7,(0) # 0 due to rounding errors in the calculation.) 


Example 5.29 A voltage e(t) is applied to the primary circuit at time ¢ = 0, and mutual induction 
M drives the current 7,(¢) in the secondary circuit of Figure 5.10. If, prior to closing 
the switch, the currents in both circuits are zero, determine the induced current i,(t) 
in the secondary circuit at time ¢ when R, = 4 Q, R, = 10 Q, L, = 2 H, L, = 8 H, 
M=2 Hand e(f) = 28 sin 2¢V. 


Figure 5.10 
Circuit of 
Example 5.29. 


Solution Applying Kirchhoff’s second law to the primary and secondary circuits respectively gives 


Rist, di, +u = = e(t) 


dt 
Rint B+uG =0 
Substituting the given values for the resistances, inductances and applied voltage leads to 
24a +2 = 28 sin 2¢ 
2S +324 10i,=0 
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5.4.2 


Figure 5.11 
Constituent elements 
of a translational 
mechanical system. 


Taking Laplace transforms and noting that 7,(0) = 7,(0) = 0 leads to the equations 


(s + 2)/,(s) + sL(s) = 


28 (5.30) 
4 


a 
sI,(s) + (4s + 5)L(s) =0 (5.31) 
Solving for /,(s) yields 


_ 28s 
(3s + 10)(s + 1)(s' +4) 
Resolving into partial fractions gives 


1,(s) = 


45 4 
L(s) =-—+— + = 
3s+10 stl 


Taking inverse Laplace transforms gives the current in the secondary circuit as 


5-26 
+2 5 
s +4 


—10¢/3 


‘ —4gq' 15 7 91 es 
in(t)=7e -7Te + 3; cos 2¢- = sin 2t 


As t > , the current will approach the sinusoidal response 


i,(t) = £ cos 2t- sin 2t 


Mechanical vibrations 


Mechanical translational systems may be used to model many situations, and involve 
three basic elements: masses (having mass M, measured in kg), springs (having spring 
stiffness K, measured in Nm’) and dampers (having damping coefficient B, measured 
in Nsm‘'). The associated variables are displacement x(f) (measured in m) and force 
F(t) (measured in N). Conventionally, the basic elements are represented symbolically 
as in Figure 5.11. 


> 4 2 ed ro 

| | \ i 

l 
F x -F | K [ FF | | F 
= — baggy —4 — <_—_ Sal 


(a) Mass (b) Spring (c) Damper 


Assuming we are dealing with ideal springs and dampers (that is, assuming that they 
behave linearly), the relationships between the forces and displacements at time f are: 


mass: F=M a3 = MX (Newton’s law) 
t 

spring: F=K(x,—~x)) (Hooke’s law) 

damper: F= a(S = a = BX, —x,) 
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Example 5.30 


Figure 5.12 
Mass-spring— damper 
system of 

Example 5.30. 


Solution 


Using these relationships leads to the system equations, which may then be analysed 
using Laplace transform techniques. 


The mass of the mass—spring—damper system of Figure 5.12(a) is subjected to an 
externally applied periodic force F(t) = 4 sin wt at time ¢ = 0. Determine the resulting 
displacement x(t) of the mass at time ¢, given that x(0) = x(0) = 0, for the two cases 


(a) @=2 (b) a=5 


In the case @ = 5, what would happen to the response if the damper were missing? 


FiQ=Kx()-Fy(t) = Bx() 


F(t) =45i 
les sie F(t) = 4sin wt 


(b) 


As indicated in Figure 5.12(b), the forces acting on the mass M are the applied force 
F(t) and the restoring forces F’, and F’, due to the spring and damper respectively. Thus, 
by Newton’s law, 


MX(t) = F(t) — Fi) — F(t) 
Since M= 1, F(t) =4 sin ot, F\(t) = Kx(t) = 25x(t) and F,(t) = Bx(t) = 6x(f), this gives 
X(t) + 6X(t) + 25x(t) = 4 sin wt (5.32) 


as the differential equation representing the motion of the system. 
Taking Laplace transforms throughout in (5.32) gives 


(s? + 6s + 25)X(s) = [sx(0) + X(0)] + 6x(0) + 


4@ 
s+o 
where X(s) is the transform of x(t). Incorporating the given initial conditions 
x(0) = x(0) = 0 leads to 


(eee (5.33) 
(s +@ )(s + 6s +25) 


In case (a), with @ = 2, (5.33) gives 


8 


2 ar Sagres ere 
(s°+.4)(s° + 6s +25) 
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which, on resolving into partial fractions, leads to 


-45+14 8s +20 
X(s = 4 ey 2 oe ee 
S) = iss peg Py 4 Ge 405 
_ 4 -4s+14, 1 8(s+3)-4 
Be aed > aay a 1G 


Taking inverse Laplace transforms gives the required response 
x(t) = 4(7 sin 2t- 4 cos 2t) + HE"(8 cos 4t - sin 41) (5.34) 


In case (b), with @= 5, (5.33) gives 


X(s) = > (5.35) 
(s' +25)(s° + 6s + 25) 
that is, 
Xs) = “iS, 1 2(s+3) +6 
s +25 °(5+3) +16 
which, on taking inverse Laplace transforms, gives the required response 
x(t) =-2 cos 51+ 4@ "(2 cos 41 +3 sin 41) (5.36) 
If the damping term were missing then (5.35) would become 
X(s)= (5.37) 
(s’ + 25) 
By Theorem 5.3, 
L{t cos 5th} = an L{cos 5t} = -2/ ; s } 
ds ls \s° +25 
that is, 
L{t cos 5t} =- |, 2s ee 
S425 (s°4+25) 8° +25 (5425) 
= ! £{sin 5t}- —e— 
(s° +25) 
Thus, by the linearity property (5.11), 
L{i sin 5t- t cos 5t} = et 
(s° +25) 


so that taking inverse Laplace transforms in (5.37) gives the response as 


x(t) = 2 (sin 5t - 5t cos 5t) 
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Example 5.31 


Figure 5.13 
Two-mass system of 
Example 5.31. 


Solution 


Because of the term ¢ cos 5t, the response x(t) is unbounded as t — oo. This arises 
because in this case the applied force F(t) = 4 sin 5¢ is in resonance with the system 
(that is, the vibrating mass), whose natural oscillating frequency is 5/2m Hz, equal to 
that of the applied force. Even in the presence of damping, the amplitude of the system 
response is maximized when the applied force is approaching resonance with the sys- 
tem. (This is left as an exercise for the reader.) In the absence of damping we have the 
limiting case of pure resonance, leading to an unbounded response. As noted in Sec- 
tion 10.10.3 of Modern Engineering Mathematics, resonance is of practical importance, 
since, for example, it can lead to large and strong structures collapsing under what 
appears to be a relatively small force. 


Consider the mechanical system of Figure 5.13(a), which consists of two masses M, = | 
and M, = 2, each attached to a fixed base by a spring, having constants K, = | and 
K, = 2 respectively, and attached to each other by a third spring having constant K, = 2. 
The system is released from rest at time ¢ = 0 in a position in which M, is displaced 
1 unit to the left of its equilibrium position and M, is displaced 2 units to the right of its 
equilibrium position. Neglecting all frictional effects, determine the positions of the 
masses at time ¢. 


Fy = K3{x2 ~ x)) 


F,= Kx; F3 = K3x2 
[ae =| 
Ye ! 
I I 


Ls, I. = =< 
x1(d) X21) ¥1(D X2(t) 


(a) {b) 


Let x,(t) and x,(t) denote the displacements of the masses M, and M, respectively from 
their equilibrium positions. Since frictional effects are neglected, the only forces acting 
on the masses are the restoring forces due to the springs, as shown in Figure 5.13(b). 
Applying Newton’s law to the motions of /, and M, respectively gives 


MX, = Fy — Fy = Kx) — x) — Kix 


MX, = —F, — F, = —Kyx.— K,Q — x) 
which, on substituting the given values for M,, M), K,, K, and K;, gives 
X, + 3x, — 2x, =0 (5.38) 
2X, + 4x, — 2x, =0 (5.39) 
Taking Laplace transforms leads to the equations 
(s° + 3)X\(s) — 2X,(s) = sx,(0) + ¥,(0) 
—X{(s) + (s? + 2)X3(s) = 5x,(0) + %,(0) 
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Since x,(t) and x,(t) denote displacements to the right of the equilibrium positions, we 
have x,(0) =—1 and x,(0) = 2. Also, the system is released from rest, so that x,(0) = x,(0) 
= 0. Incorporating these initial conditions, the transformed equations become 


(s* + 3)X((s) — 2X,(s) = —s (5.40) 

—=X,(s) + (s* + 2)X,(s) = 2s (5.41) 
Hence 

L@< Is +5s 


(s° +4)(s° +1) 


Resolving into partial fractions gives 


S Ss 
+ 


s4+1 44 


X,(s) = 


which, on taking inverse Laplace transforms, leads to the response 


x,(t) = cost + cos 2t 


Substituting for x,(t) in (5.39) gives 


X(t) = 2x,(t) + X(t) 


=2cost+2cos2t—cost—4cos 2t 


that is, 


x(t) = cos t— 2 cos 2t 


Thus the positions of the masses at time ¢ are 


x(t) = cos t— 2 cos 21, 


5.4.3 Exercises 


x(t) = cos t+ cos 2t 


Check your answers using MATLAB or MAPLE whenever possible. 


Use the Laplace transform technique to find the 
transforms /,(s) and L,(s) of the respective currents 
flowing in the circuit of Figure 5.14, where i,(f) is 
that through the capacitor and i,(¢) that through 


50yF 


A 


t=0 


E sin 100: 100Q 


Figure 5.14 Circuit of Exercise 7. 


the resistance. Hence, determine 7,(t). (Initially, 
i,(0) = i,(0) = q,(0) = 0.) Sketch ,(¢) for large 
values of ¢. 


At time ¢ = 0, with no currents flowing, a voltage 
v(t) = 10 sin ¢ is applied to the primary circuit of 
a transformer that has a mutual inductance of 1 H, 
as shown in Figure 5.15. Denoting the current 
flowing at time ¢ in the secondary circuit by i,(4), 
show that 


10s 
(s° +75 +6)(s°' +1) 


L{in(t)} = 
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er | 20 


v(t) = 10 sint 


M=i1H 
Figure 5.15 Circuit of Exercise 8. 


and deduce that 


: . -6 . 
i(t)=-e'+¥e ‘+3 cost +3 sint 


In the circuit of Figure 5.16 there is no energy 
stored (that is, there is no charge on the capacitors 
and no current flowing in the inductances) prior to 
the closure of the switch at time t = 0. Determine 
i,(t) for t > 0 for a constant applied voltage 

Ey =10V. 


Figure 5.16 Circuit of Exercise 9. 


Determine the displacements of the masses /, and 
M, in Figure 5.13 at time t > 0 when 


M,=M,=1 


K,=1,K)=3 and K,=9 


What are the natural frequencies of the 
system? 


When testing the landing-gear unit of a space 
vehicle, drop tests are carried out. Figure 5.17 is a 
schematic model of the unit at the instant when it 
first touches the ground. At this instant the spring 
is fully extended and the velocity of the mass is 
\(2gh), where h is the height from which the 

unit has been dropped. Obtain the equation 
representing the displacement of the mass at 

time ¢ > 0 when M = 50 kg, B= 180 N sm! and 
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Figure 5.17 Landing-gear of Exercise 11. 


K = 474.5 Nm’, and investigate the effects of 
different dropping heights /. (g is the acceleration 
due to gravity, and may be taken as 9.8 ms~”.) 


Consider the mass—spring—damper system 
of Figure 5.18, which may be subject to two 
input forces u,(t) and w,(t). Show that the 
displacements x,(t) and x,(t) of the two masses 
are given by 


X(t) 


Figure 5.18 Mechanical system of Exercise 12. 


Ms’ +B,s +K> u, 


= (s) +48 vo} 


x(t) = ¢'| 


M,s° +B,s +K, v.)} 


x(t) = |B U\(s) + x 


where 


A=(M,s? + Bis + K,)(M,s? + Bys + Ky) — Bes 
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Step and impulse functions 


5.5.1 The Heaviside step function 


Figure 5.19 
Heaviside unit 
step function. 


Figure 5.20 
Piecewise-continuous 
function. 


In Sections 5.3 and 5.4 we considered linear differential equations in which the forcing 
functions were continuous. In many engineering applications the forcing function may 
frequently be discontinuous, for example a square wave resulting from an on/off 
switch. In order to accommodate such discontinuous functions, we use the Heaviside 


unit step function H(t), which, as we saw in Section 5.2.1, is defined by 
< 

H(t) = 0 (t<0) 

1 (t20) 


and is illustrated graphically in Figure 5.19(a). The Heaviside function is also fre- 
quently referred to simply as the unit step function. A function representing a unit step 
at t = a may be obtained by a horizontal translation of duration a. This is depicted 
graphically in Figure 5.19(b), and defined by 


0 (t<a) 


He-ay={' (t= a) 


H(t) 


(a) 


The product function f/(t)H(t — a) takes values 
0 (t <a) 
f(t) (t= a) 


so the function H(t — a) may be interpreted as a device for ‘switching on’ the function 


foo «)= 


f(t) at t= a. In this way the unit step function may be used to write a concise formula- 


tion of piecewise-continuous functions. To illustrate this, consider the piecewise- 
continuous function f(t) illustrated in Figure 5.20 and defined by 


F(t) 
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Top hat function. 
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A@ (0=t<4fh) 
SO =) (tf St < th) 
AQ) (t= h) 
To construct this function f(t), we could use the following ‘switching’ operations: 


(a) switch on the function f(t) at t= 0; 
(b) switch on the function f,(t) at f = ¢, and at the same time switch off the function 


Ai); 


(c) switch on the function f(t) at f = ft, and at the same time switch off the function 
ft). 


In terms of the unit step function, the function f(t) may thus be expressed as 


f(t) =AOAD + [AM -AOIG - 0) + LAO - AOI tb) 
Alternatively, f(#) may be constructed using the top hat function H(t — a) — H(t — b). 
Clearly, 

1 (a<t<b) 


; (5.42) 
0 otherwise 


H(t-a)- H(t-b) = 


which, as illustrated in Figure 5.21, gives 
t <t<b 
folate a) HEr~ by) =O (a ) 


0 otherwise 


H(t—a)— H(t —-b) 


0 -—— 
Using this approach, the function f(t) of Figure 5.20 may be expressed as 
ft) = (OLA) — Ht - t,)] + KOLA 4) — H(t - 4)1 + A(DH(t- 6) 
giving, as before, 
f(t) = f(OH(t) + LAD) — ACMA — 1) + LAO — ADI - 4) 
It is easily checked that this corresponds to the given formulation, since for 0 < ¢ < ¢, 
A(t) =1, A(t-t,)=AH(t-t)=0 
giving 
fO=fO O<t<t) 
while fort; <t<t, 
A(t)= A(t—t,) =1, A(t — tr) =0 
giving 


FO=AO+AO-AOI=AO Gh St <4) 
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and finally for ¢ = ¢, 
A(t)= H(t-—t,) = HA(t-t)=1 
giving 


SO =AO +1AO-AOl+ (AO -AOI=AO > 4) 


Example 5.32 Express in terms of unit step functions the piecewise-continuous causal function 


2 


2t (0 <t< 3) 
f(t) = 5t+4 (3<t<5) 
9 (t = 5) 
Figure 5.22 f() 
Piecewise-continuous 


function of 
Example 5.32. 


Solution f(t) is depicted graphically in Figure 5.22, and in terms of unit step functions it may be 
expressed as 


f(t) = 2PH(t) + (t+ 4 - 20)A(t- 3) + (9 -t-4H(t— 5) 
That is, 
f(t) = 2?H(t) + (44 t- 20’)A(t- 3) + (5 — DH(t—- 5) 


Example 5.33 Express in terms of unit step functions the piecewise-continuous causal function 


0 (t< 1) 
tl (=F<3) 
fO=33 GSr<5) 
2 (5<t< 6) 

0 (t= 6) 


Solution f(t) is depicted graphically in Figure 5.23, and in terms of unit step functions it may be 
expressed as 


f(t) = 1H(t- 1) +B — )H(t— 3) + (2 —3)H(t— 5) + (0 — 2)H(t- 6) 
That is, 
f(t) = 1H(t— 1) + 2H(t— 3) — 1H(t— 5) — 2H(t- 6) 
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Figure 5.23 


Piecewise-continuous 


function of 
Example 5.33. 


fo 


5.5.2 Laplace transform of unit step function 


By definition of the Laplace transform, the transform of H(t — a), a = 0, is given by 


L{H(t- a)} 


That is, 


AH(t- ayy = = (a= 0) 


and in the particular case of a = 0 


LLH(t)} = - 


| H(t-a)e“'dt = | crane | le“'dt 
0 0 a 


(5.43) 


(5.44) 


This may be implemented in MATLAB using the commands 


syms s t 
H=sym(‘Heaviside(t)’) 
laplace (H) 


which return 


ans=1/s 


It may also be obtained directly using the command 


laplace(sym(‘Heaviside(t) ’) ) 


Likewise to obtain the Laplace transform of H(t-2) we use the commands 


H2=sym( ‘Heaviside (t-2)’) 


laplace(H2) 
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which return 
ans=exp (-2*s)/s 
In MAPLE the results are obtained using the commands: 


with(inttrans) : 
laplace (Heaviside(t),t,s); 
laplace (Heaviside(t-2),t,s); 


Example 5.34 Determine the Laplace transform of the rectangular pulse 


0 (t<a) 
I(t) =)K (ax<t<b) Kconstant, b>a>0 
0 (t=b) 


Solution The pulse is depicted graphically in Figure 5.24. In terms of unit step functions, it may 
be expressed, using the top hat function, as 


LO 
K — f(t) = K[H(¢— a) — H(t— 6)] 
— Then, taking Laplace transforms, 


Oo 
coe L{flt)} = KAM(t a)} — KL{H(t— b)} 
Figure 5.24 . ; : 
Rectangular pulse. which, on using the result (5.24), gives 


-bs 


Li f()}=K e Ke 
S S 
That is, 


LO} = 7 Cea 


Example 5.35 Determine the Laplace transform of the piecewise-constant function f(f) shown in 
Figure 5.23. 


Solution From Example 5.33 f(t) may be expressed as 
f(t) = 1H(t— 1) + 2H(t- 3) -— 1M(t- 5) — 2H(t - 6) 


Taking Laplace transforms, 
LE f(t)} = 1L{H(t — 1)} + 2L{A(t — 3)} - LLL A(t - 5)} -— 2L{ A(t - 6)} 
which, on using the result (5.43), gives 
-s -3s -5s 
LA f(Hy=+22 -£ -22—- 
s s s s 


That is, 


Lf ()}= te" iene aa) 
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2) Check that the same answer is obtained using the MATLAB sequence of commands 


symsst 
Hl=sym(‘Heaviside(t-1)’); 
H3=sym(‘Heaviside(t-3)’); 
H5=sym(‘Heaviside(t-5)’); 
H6=sym( ‘Heaviside (t-6)’); 
laplace (H1-2*H3-H5-2*H6) 
In MAPLE the commands 
with(inttrans) : 
laplace (Heaviside (t-1)+Heaviside(t-3)*2 - Heaviside(t-5) 


= lsleehyalshicla (ie 6) "4 ie, Ss) p 


return the answer 


Gl gee 


ef) — 996) 


Ss 


5.5.3. The second shift theorem 


This theorem is dual to the first shift theorem given as Theorem 5.2, and is sometimes 
referred to as the Heaviside or delay theorem. 


Li ft — a)H(t — a)} = &“F(s) 


Proof By definition, 


L{ f(t - a)H(t- a)} = | 


co 


0 


Making the substitution T= t— a, 


LY f(t- a)H(t- a)} = | 


oy 


0 


Theorem 5.4 If L{ f(t)} = F(s) then for a positive constant a 


f(t-a)H(t-a)e“ dt 


f(t-a)je"dt 


“| 


f(T) aT 


= “| Te ar 


co 


Since F(s) = Lf f(t)} = | f(T) e", it follows that 


Lift — a)H(t — a)} = €F(s) 
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It is important to distinguish between the two functions f(MH(t — a) and f(t— a)H(t— a). 
As we saw earlier, f(t)H(t — a) simply indicates that the function f(t) is ‘switched on’ 
at time t= a, so that 
0 (t <a) 
ff) (ta) 


On the other hand, f(t — a)H(t— a) represents a translation of the function f(t) by a units 
to the right (to the right, since a > 0), so that 


fOH(t- a) = | 


f(t-a)H(t~a) = \" oie: 

f(t-a) (t= a) 
The difference between the two is illustrated graphically in Figure 5.25. f(t — a)H(t — a) 
may be interpreted as representing the function f(t) delayed in time by a units. Thus, when 
considering its Laplace transform e“*F(s), where F(s) denotes the Laplace transform of 
f(t), the component e“ may be interpreted as a delay operator on the transform F(s), 
indicating that the response of the system characterized by F(s) will be delayed in time 
by a units. Since many practically important systems have some form of delay inherent 
in their behaviour, it is clear that the result of this theorem is very useful. 


f(A) fH - a) ft-a)H(t-a) 


0 a t oO a t oO a t 
Figure 5.25 Illustration of f(t — a)H(t— a). 


Example 5.36 Determine the Laplace transform of the causal function f(t) defined by 


Solution f(t) is illustrated graphically in Figure 5.26, and is seen to characterize a sawtooth pulse 
of duration 5. In terms of unit step functions, 


a S(t) = tH(t) — tH(t — b) 


b In order to apply the second shift theorem, each term must be rearranged to be of the 
form f(t — a)H(t — a); that is, the time argument t — a of the function must be the same 
as that of the associated step function. In this particular example this gives 


e b t f(t) = tH(t) — (t— b)H(t — b) - bH(t — b) 


Figure 5.26 
Sawtooth pulse. 


Taking Laplace transforms, 


AKO} = LtH()} — L(t— b)H(t— b)} — bAAA(t— b)} 


www.2Ofile.org 


Example 5.37 


Solution 
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which, on using Theorem 5.4, leads to 


1 -bs ee’ _ 1 a -bs 
Ao aa-6 2b = = = = =a 
Ss 5 g r - 
giving 
Asfo}y=4a-e%)-2e™ 
s Ss 


It should be noted that this result could have been obtained without the use of the 
second shift theorem, since, directly from the definition of the Laplace transform, 


Lf} = | fje dt = | te" dt+ | Oe" ds 


te” b b ee”! te” ee g 
cere a es 
0 0 S Jo 


as before. 


Obtain the Laplace transform of the piecewise-continuous causal function 


4 8 8(0=7< 3) 
f@ = 5t+4 GB <t<5) 
9 (t = 5) 


considered in Example 5.32. 


In Example 5.32 we saw that f(t) may be expressed in terms of unit step functions as 


f(t) = 27°H(t) — (202 — t— 4)H(t - 3) — (t- 5) H(t — 5) 


Before we can find Y{ f(t)}, the function 27? — t — 4 must be expressed as a function of 
t — 3. This may be readily achieved as follows. Let z = t — 3. Then 


20? —t-4=2(2+3)Y -(z+3)-4 
= 277+ 1lz+11 
= 2(t- 3) + 11(¢-3)+ 11 
Hence 


f(t) = 2PH(t) — [20¢- 3Y + 11(¢— 3) + IJ — 3) - (¢— 5) H(t 5) 


Taking Laplace transforms, 
SLi f(t)} = 2A{V A(t) — L{[2(t- 3 + 11(t- 3) + 1A - 3) 
— £L{(t— 5)H(t — 5)} 
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which, on using Theorem 5.4, leads to 


22-6 £20 + 11t+ 11} -e Lt} 
S 


-5s 
=4-07( 44H pM) 2 
Ss Ss 


Af} 


“7 2 
Ss s s 


Again this result could have been obtained directly from the definition of the Laplace 
transform, but in this case the required integration by parts is a little more tedious. 


(2) Having set up s and t as symbolic variables and specified H, H1 and H5 then the 
MATLAB commands 


UgjOLace (22 (Ber DA) al) — (ie='5)) IS) p 

pretty (ans) 
generate 

ans= 4/s*-1lexp(-3s) /s-l1lexp(-3s) /s*-4exp (-3s) /s*-exp(-5s) /s’ 
In MAPLE the commands 


with(inttrans) : 
lap lacel(deaviside(t) *2*t 2) sheavismdel(e—s)s(2 to? 4) 
= Keene (ie=5))*? (ic=5) pic, 8) ¢ 


return the answer 


Ree (2s oP Gile® + lis. ay 
Se S 


3 


5.5.4 Inversion using the second shift theorem 


We have seen in Examples 5.34 and 5.35 that, to obtain the Laplace transforms of 
piecewise-continuous functions, use of the second shift theorem could be avoided, 
since it is possible to obtain such transforms directly from the definition of the Laplace 
transform. 

In practice, the importance of the theorem lies in determining inverse transforms, 
since, as indicated earlier, delays are inherent in most practical systems and engineers 
are interested in knowing how these influence the system response. Consequently, by 
far the most useful form of the second shift theorem is 


L\feF(s)} = fit — a)H(t — a) (5.45) 


Comparing (5.45) with the result (5.12), namely 
L'F(s)} = f(A(t) 
we see that 


L'.e“F(s)} =[f(QA(t)] with t replaced by t- a 
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Example 5.38 


Solution 


EJ 
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indicating that the response f(t) has been delayed in time by a units. This is why the 
theorem is sometimes called the delay theorem. 


This is readily implemented in MATLAB using the command ilaplace. 


-4s 
Determine ¥' ae } 


This may be written as L'{e““F(s)}, where 


= 4 
S(s +2) 


F(s) 
First we obtain the inverse transform f(t) of F(s). Resolving into partial fractions, 


2 2 
F(s) = =-— 
(s) Ss st2 
which, on inversion, gives 
f=2-2e 


a graph of which is shown in Figure 5.27(a). Then, using (5.45), we have 


=i |) ay 4 _ oep-ly.-4s = = _ 
A fe zial-2 {e “F(s)} = f(t-4)A(t- 4) 
=(2-2e°) A(t — 4) 


giving 


v{ Aen L ‘ (<4) 
s(s+2) 7 2-68 (t = 4) 


which is plotted in Figure 5.27(b). 


Using MATLAB confirm that the commands 


ilaplace (4*exp(-4*s)/(s*(s+2))); 
pretty (ans) 


generate the answer 
lel (je!) (ergo) (= Ze 455}) )) 


The same answer is obtained in MAPLE using the commands 


with(inttrans): 
invlaplace (4*exp (—4*s)//(s* (s#2)),s,t&); 
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Figure 5.27 Inverse 
transforms of 
Example 5.38. 


Example 5.39 


Solution 


fO 


o | 2 3 4 5 6 7 


(a) Graph of f(1) 
f(t —)H(t - 4) 
2 


O 1 2 3 4 5 6 7 


(b) Graph of f(t - 4)A(t - 4) 
Determine #' 2 - == : 
s(s +1) 


This may be written as £"'{e“"F(s)}, where 


S+3 


Fs) = 
s(s' +1) 


Resolving into partial fractions, 


Fg) =2- 4 
s stl s+] 


which, on inversion, gives 
f(t) =3 -3cost+ sint 
a graph of which is shown in Figure 5.28(a). Then, using (5.45), we have 


-je"(s +3 
ca ( 
s(s +1) 


= L£'{e"F(s)} = f(t-m)H(t- 7) 


= [3 —3cos(t— 7m) + sin(t— 7)|A(t — 7) 


= (3+ 3cost-— sint)H(t— 7) 
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f() 


-5 


(a) 
f(t — n)H(t — 1) 
10 


-5 
10 12 14 ¢t 


oO 
N 
a 

= 
a 
oo 


(b) 


Figure 5.28 Inverse transforms of Example 5.39. 


giving 


a }={' (t< 7) 


s(s’ +1) 34+3cost-sint (t=) 


which is plotted in Figure 5.28(b). 


5.5.5 Differential equations 


We now return to the solution of linear differential equations for which the forcing 
function f(f) is piecewise-continuous, like that illustrated in Figure 5.20. One 
approach to solving a differential equation having such a forcing function is to solve 
it separately for each of the continuous components /,(t), 4(¢), and so on, comprising 
f(t), using the fact that in this equation all the derivatives, except the highest, must 
remain continuous so that values at the point of discontinuity provide the initial con- 
ditions for the next section. This approach is obviously rather tedious, and a much 
more direct one is to make use of Heaviside step functions to specify f(t). Then the 
method of solution follows that used in Section 5.3, and we shall simply illustrate it 
by examples. 
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Example 5.40 


Solution 


Method 1 


Obtain the solution x(t), t = 0, of the differential equation 


dx dx _ 
—>+5—=+ 6x = f(t) (5.46) 
dt dt 


where f(t) is the pulse function 


3 = 
ft) = {e eee 


and subject to the initial conditions x(0) = 0 and x(0) = 2. 


To illustrate the advantage of using a step function formulation of the forcing function 


f(t), we shall first solve separately for each of the time ranges. 


For 0 S ¢ < 6, (5.46) becomes 


2 
dx +5 de + 6x =3 
dt dt 
with x(0) = 0 and x(0) = 2. 
Taking Laplace transforms gives 


(s? + 59 + 6)X(s) = sx(0) +2(0) + 5x(0) + ; oe ; 


That is, 


2s+3 ae 1 
XG) = —= 1 74 2 
S(s+2)(st+3) gs gst+2 s+3 


which, on inversion, gives 
x(th=3the"-e"% (0<t<6) 


We now determine the values of x(6) and x(6) in order to provide the initial conditions 
for the next stage: 


x(6)=$+he"-e =a, x6)=-e"+3e%=8 


For t = 6 we make the change of independent variable T= t — 6, whence (5.46) becomes 
2 
Ox 15 OX 4 6 =9 
dT dT 


subject to x(T = 0) = a and x(T= 0) = B. 
Taking Laplace transforms gives 


(s? + 5s + 6)X(s) = sx(T = 0) + X(T = 0) + 5x(T = 0) = as + 5a+ B 
That is, 


XG) = as+Sa+B  Bt+3a_ Bt+2a 


~ (s+2)(9+3) st+2 943 
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Figure 5.29 
Forcing function 
and response of 
Example 5.40. 


Method 2 
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which, on taking inverse transforms, gives 
x(T) = (B+ 3a)e*" —(B+2ae*" 

Substituting the values of a and f and reverting to the independent variable ¢ gives 
x= (3+ fe %er9-U+e%e% ¢= 6) 

That is, 
x(t) = (fee) + (Ge%™ —e) (¢ > 6) 

Thus the solution of the differential equation is 


tthe” 
x(t) = 


1 .-2¢ -3t 3 _-2(t-6) -3(t-6) 
7e€° -e )+(Ge -e ) 


t 3t 


-e (0<t< 6) 
(t > 6) 


The forcing function f(t) and response x(t) are shown in Figures 5.29(a) and (b) 
respectively. 


SOR x(t) A 
eae 
6-+ 
o4+ 
——— 
0.2 
a ae ee er — a ee 
0 3 6 9 2 t OF 2 4 6 8 W0F 
(a) (b) 


In terms of Heaviside step functions, 
f(t) = 3H(t) — 3H(t - 6) 
so that, using (5.43), 


as, 
Li f(t} = 3-20 
S AY 
Taking Laplace transforms in (5.46) then gives 
(s? + 5s + 6)X(s) = sx(0) + X(0) + 5x(0) + LLCO} = 242-30 
Ss Ss 
That is, 


Xs) = 254+3 es 3 
s(s+2)(s+3) s(s+2)(s+3) 


1 3 
I Care gre) acl ee = 
S st2 st+3 Ss st+2 st+3 


Taking inverse Laplace transforms and using the result (5.45) gives 


x(t) = (5 + 307% %)— (5 — 507 + A(t 6) 


www.2Ofile.org 


406 LAPLACE TRANSFORMS 


which is the required solution. This corresponds to that obtained in Method 1, since, 
using the definition of H(t — 6), it may be written as 


eae e” (0 < t <6) 
(ie a + @G et) _ gciey (t = 6) 


This approach is clearly less tedious, since the initial conditions at the discontinuities 
are automatically taken account of in the solution. 


(2) It seems that the standard dsolve command is unable to deal with differential 
equations having such Heaviside functions as their forcing function. To resolve this 
problem use can be made of the maple command in MATLAB, which lets us 
access MAPLE commands directly. Confirm that the following commands produce 
the correct solution: 


upaolle( 7 Clas =claiiei (S<(ic)) , eS 2) ea oelseie (oe) pic.) 4h xe (ic) 
=3*Heaviside-3*Heaviside(t-6);') 

ans= 

Ge ge GPE (Ge(ie) , 9S" (iE, 2) ) eS eCaLIE (Ge (IE) 18) O52 (1e) 


= 3*Heaviside-3*Heaviside(t-6) 
maple (‘dsolve({de,x(0)=0,D(x) (0)=2},x(t)),method=laplace; ’) 


In MAPLE the answer may be obtained directly using the commands: 


with(inttrans) : 

Glegachbiit (Se (ie) pic S2)) sea Caliew (Se (iE) pe) Hass (ie) 
-3*Heaviside-3*Heaviside(t-6) ; 

dsolve({de,x(0)=0,D(x) (0)=2},x(t)),method=laplace; 


Example 5.41 —_ Determine the solution x(t), ¢ = 0, of the differential equation 


fr 4284 5x = f(s) (5.47) 
where 
_ jt (O0StS7) 
fo= {i (t= 7) 


and subject to the initial conditions x(0) = 0 and x(0) = 3. 


Solution Following the procedures of Example 5.36, we have 
S()) = tH(t) — tH(t — 7) 
= tH(t) — (t— n)H(t — 1) — NH(t — 7) 


so that, using Theorem 5.4, 


“1s -TSs 


Lisp} 4-&>- B= bem ( 448) 
S S 


Ss AY Ss 
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Taking Laplace transforms in (5.47) then gives 
(s? + 2s + 5)X(s) = sx(0) + X(0) + 2x(0) + Lf f(t) 


using the given initial conditions. 
Thus 


35°+1 _e™ l+st 


X() = +3 rE 
s(s +2s+5) s(s' +2s+5) 


which, on resolving into partial fractions, leads to 


Xs) st 2+5e a 


s (s+1)+ 25) s 3s (s+1)+4 


_ aaa ae 


2g = @Ely +4 


25| ss (s+1)°+4 


- a _ (Sn -2)(s eo 
Taking inverse Laplace transforms and using (5.45) gives the desired solution: 
x(t) = = (-2 + 5t+ 2e“ cos 2t + 36” sin 22) 
— £[(Sm— 2) + 5(¢— m) — (Sn — 2) &™ cos 2(t- m) 
— $(50 +3) e™ sin 2(t- m)JA(t— 7) 
That is, 
x(t) = & [5t— 2+ 2e“(cos 2t+ 18 sin 2A)] 
— x {5t-2- ee [(5m — 2) cos 2r+ 1 (5m + 3) sin 24]}H(t — 7) 
or, in alternative form, 


- anode sin2t)] (0<t<n) 
a = 


A e'{(2 + (Sm - 2) e”) cos 2¢+[36+4(5n+3)e"] sin 2t} (t =n) 


5.5.6 Periodic functions 


We have already determined the Laplace transforms of periodic functions, such 
as sin wt and cos wt, which are smooth (differentiable) continuous functions. In many 
engineering applications, however, one frequently encounters periodic functions that 
exhibit discontinuous behaviour. Examples of typical periodic functions of practical 
importance are shown in Figure 5.30. 

Such periodic functions may be represented as infinite series of terms involving step 
functions; once expressed in such a form, the result (5.43) may then be used to obtain 
their Laplace transforms. 
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Figure 5.30 fo 
Typical practically K 
important periodic 
functions: (a) square 
wave; (b) sawtooth 
wave; (c) repeated 
pulse wave; (d) half- 
wave rectifier. -K 


F(t) 


Example 5.42 Obtain the Laplace transform of the square wave illustrated in Figure 5.30(a). 


Solution In terms of step functions, the square wave may be expressed in the form 
f(t) = KH(t) - 2KH(t- } T) + 2KH(t- T) - 2KH(t- 3T) + 2KH(t-2T) +... 
= K[H(t) - 2H(t- 4 T) + 2H(t- T) - 2H@¢- 27) + 2H(t-2T)+...] 


Taking Laplace transforms and using the result (5.43) gives 
Lf ()} = F(s) = x(? _ 2 972 2 oT 2 382 2 oT ee. } 
ss s s s 


_ ZR et 4 (ery? _ (e257 4 (Cae ery 


ileal 
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The series inside the square brackets is an infinite geometric progression with first 
term 1 and common ratio —e~*"”, and therefore has sum (1 + e*””)'. Thus, 


2K 1 _K_Ki-e" 


F(s)= 
Sf4er? sg S14eo" 


That is, 


LI f(t)} = F(s) =F tanh L917 


The approach used in Example 5.42 may be used to prove the following theorem, which 
provides an explicit expression for the Laplace transform of a periodic function. 


Theorem 5.5 If f(t), defined for all positive t, is a periodic function with period 7, that is 
f(t+nT) = f(t) for all integers n, then 


1 
l-e° 


Lf flt)} = : | e fay d 


0 


Proof If, as illustrated in Figure 5.31, the periodic function f(t) is piecewise-continuous over 
an interval of length 7, then its Laplace transform exists and can be expressed as a 
series of integrals over successive periods; that is, 


°° 


xinr= | f(t)e™' dt 


0 


= | foe"ars | gears | fide“ dt+... 


0 T 2T 


nT 

+ | f(thedt+... 
(n-1)T 

If in successive integrals we make the substitutions 


t=T+nT (n=0,1,2,3,...) 


then 


Li f(t)}= >| f(tt nT) err dr 


n=0 


Figure 5.31 ff) 
Periodic function 
having period 7. 
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fi 


Figure 5.32 

Plot of periodic 
function within one 
period. 


Example 5.43 


Solution 


Since f(t) is periodic with period 7, 
f(tt+nT)=f(t) (n=0, 1, 2,3,...) 


so that 
os T a T 
Afor= > | f(tye* edt = » | f(t) edt 
n=0 4 0 n=0 0 
The series Dp O°" =1L+e° +e°7 +e°"+... is an infinite geometric progression 


with first term 1 and common ratio e~’. Its sum is given by (1 — e*’)', so that 


HMO} =- | ft) "dt 
sn 


Since, within the integral, tT is a ‘dummy’ variable, it may be replaced by f¢ to give the 
desired result. 


end of theorem 


We note that, in terms of the Heaviside step function, Theorem 5.5 may be stated as 
follows: 


If f(t), defined for all positive ¢, is a periodic function with period 7 and 
Ait) =f — HE T)) 


then 


Af} = -e N AAO} 


This formulation follows since f(t) is periodic and f(t) = 0 for ¢ > T. For the periodic 
function f(t) shown in Figure 5.31 the corresponding function f{(f) is shown in 
Figure 5.32. We shall see from the following examples that this formulation simplifies 
the process of obtaining Laplace transforms of periodic functions. 


Confirm the result obtained in Example 5.42 using Theorem 5.5. 


For the square wave f(f) illustrated in Figure 5.30(a), f(t) is defined over the period 
0<t<Tby 


K (<< 57) 


40 =| 
ke Urarey) 


Hence we can write f(t) = K[H(t) — 2H(t - iT) + A(t — T)], and thus 


na eae =x(2 2491? +2e7) - Kq ~My? 
ss Ss s 
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Using the result of Theorem 5.5, 
K( _ Male Kd _ sales 
Li f(t} 7 : -sT. - = -sT/2 
s(l-e ) s(l-e “)\(1+e"%) 


-sT/2 
=Kl-e — =F tanh ist 
sl+e° 8 


confirming the result obtained in Example 5.42. 


Example 5.44 Determine the Laplace transform of the rectified half-wave defined by 


13 


14 


ios sin@t (0<t<7n/o) 
. 0 (t/@ <t < 2m/@) 


f(t+2nt/o)=f(t) for all integers n 


Solution f(¢) is illustrated in Figure 5.30(d), with T= 21/@. We can express f(t) as 
f(t) = sin wt [A(t) — A(t — 1/@)] 
= sin otH(t) + sina@(t — t/@)H(t — 1/@) 


So 


@ -stt/ @ @ -sTt/ 
Li fi(O}= 7 7 te os = (140) 
S+@ S+@ Ss +@ 


Then, by the result of Theorem 5.5, 


Ce Ol eee ® 
S 


24g i= eon (s° + wo )(1 _ eo) 


5.5.7. Exercises 


Check your answers using MATLAB or MAPLE whenever possible. 


A function f(t) is defined by af (0<t<4) 
t (0<t<1) (a) f(t) =42t-3 (4<1t<6) 
Y= 10 sy 5 (t>6) 
Express f(t) in terms of Heaviside unit step f (0s7¢<1) 
functions and show that (b) g(t) =42-t (1<t<2) 
0 (t > 2) 


Lifpy=4-e%)-+e* 
Ss Ss 


15 Obtain the inverse Laplace transforms of the 


Express in terms of Heaviside unit step functions the following: 

following piecewise-continuous causal functions. - a 

In each case obtain the Laplace transform of the (a) ; (b) i 
function. (s-2) (s+3)(st 1) 
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16 


17 


18 


19 


20 


+1 ee: +1 -1sS 
(c) —e (d) —“*—e 
s(s +1) s +st+1] 
(e) = S__ godnsis e 1ne 
sg) +25 s(s' +1) 


Given that x = 0 when ¢ = 0, obtain the solution of 
the differential equation 

dx 

=+x=f(t t=0 
a fit) ( ) 
where f(t) is the function defined in Exercise 13. 
Sketch a graph of the solution. 


Given that x = 1 and dx/dt = 0, obtain the solution 
of the differential equation 

dx yy = g(r) 
dt dt 


where g(t) is the piecewise-continuous function 
defined in Exercise 14(b). 


Show that the function 
0 (0 <1t<n) 
Ao={? —— 
sint (¢t = 5%) 
may be expressed in the form f(t) = cos (t — i) 
A(t— 11), where H(t) is the Heaviside unit step 
function. Hence solve the differential equation 
2 
dx 38 yoy = f(1) 
dt dt 
where f(t) is given above, and x = 1 and 
dx/dt = —1 when t= 0. 


Express the function 
3 0Oxt<4 
At= , 
2t-5 (t24) 
in terms of Heaviside unit step functions and obtain 
its Laplace transform. Obtain the response of the 
harmonic oscillator 
¥+x=f(t) 
to such a forcing function, given that x = 1 and 
dx/dt = 0 when t= 0. 


The response 0,(t) of a system to a forcing function 
O(t) is determined by the second-order differential 
equation 

6,+ 66,+100,=0, (¢= 0) 
Suppose that 6,(t) is a constant stimulus applied for 
a limited period and characterized by 


aco = (0<t<a) 
0 (t2a) 


21 


Ze 


28) 


24 


Determine the response of the system at time ¢ 
given that the system was initially in a quiescent 
state. Show that the transient response at time 
T (> aj)is 

-2e%cos T+ 3 sin T— e*“[cos(T— a) 


+3 sin (T—-a)]} 


The input 6+) and output 6,(t) of a servomechanism 
are related by the differential equation 


6,+80,+160,=0, (t=0) 
and initially 6,(0) = 6,(0) = 0. For 6; = f(t), where 


l-t (0<t<1) 


Ao=|" (t >1) 


Show that 


Foy} = +4er 
S S 


and hence obtain an expression for the response of 
the system at time ¢. 


During the time interval ¢, to 4, a constant 
electromotive force e) acts on the series RC circuit 
shown in Figure 5.33. Assuming that the circuit is 
initially in a quiescent state, show that the current 
in the circuit at time ¢ is 


=(t-t))/RC =(t-t)/RC 


i() = Fle H(t-t,)-e H(t~ t)] 


Sketch this as a function of time. 


R Cc 


e(t) i i(t) 


Figure 5.33 Circuit of Exercise 22. 


A periodic function f(t), with period 4 units, is 
defined within the interval 0 = ¢ < 4 by 


31 (0<t<2) 


Ko={t (2<t<4) 


Sketch a graph of the function for 0 < t < 12 and 
obtain its Laplace transform. 


Obtain the Laplace transform of the periodic 
sawtooth wave with period 7, illustrated in 
Figure 5.30(b). 
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Figure 5.34 
Impulse function. 


5.5 STEP AND IMPULSE FUNCTIONS 413 


The impulse function 


Suppose a hammer is used to strike a nail then the hammer will be in contact with the 
nail for a very short period of time, indeed almost instantaneously. A similar situation 
arises when a golfer strikes a golf ball. In both cases the force applied, during this short 
period of time, builds up rapidly to a large value and then rapidly decreases to zero. 
Such short sharp forces are known as impulsive forces and are of interest in many 
engineering applications. In practice it is not the duration of contact that is important 
but the momentum transmitted, this being proportional to the time integral of the force 
applied. Mathematically such forcing functions are represented by the impulse function. 
To develop a mathematical formulation of the impulse function and obtain some insight 
into its physical interpretation, consider the pulse function @(t) defined by 


0 (0<t<a-iT) 
ot) =) A/T (a-}T<t<a+tiT) 
0 (t=a+iT) 


and illustrated in Figure 5.34(a). Since the height of the pulse is 4/7 and its duration (or 
width) is 7, the area under the pulse is A; that is, 


Pe at+T/2 P 
g(t) dt = | —dt=A 
/ a-T/2 r 


If we now consider the limiting process in which the duration of the pulse approaches 
zero, in such a way that the area under the pulse remains 4, then we obtain a formula- 
tion of the impulse function of magnitude A occurring at time ¢ = a. It is important to 
appreciate that the magnitude of the impulse function is measured by its area. 

The impulse function whose magnitude is unity is called the unit impulse function 
or Dirac delta function (or simply delta function). The unit impulse occurring at 
t = a is the limiting case of the pulse @(¢) of Figure 5.34(a) with A having the value 
unity. It is denoted by d(¢ — a) and has the properties 


d(t-—a)=0 (t#¥a) 
| O(t-a)dt=1 


Likewise, an impulse function of magnitude A occurring at t= a is denoted by Ad(t— a) 
and may be represented diagrammatically as in Figure 5.34(b). 

An impulse function is not a function in the usual sense, but is an example of a class 
of what are called generalized functions, which may be analysed using the theory of 


g(t) Ad(t—a) 


a-!Taa+3T 1 a t 


(a) (b) 
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$1) 


T 
O a~T a a+T t fT O -T 1! 
Figure 5.35 Approximation to a unit pulse. Figure 5.36 Pulse at the origin. 


generalized calculus. (It may also be regarded mathematically as a distribution and 
investigated using the theory of distributions.) However, its properties are such that, 
used with care, it can lead to results that have physical or practical significance and 
which in many cases cannot be obtained by any other method. In this context it provides 
engineers with an important mathematical tool. Although, clearly, an impulse function 
is not physically realizable, it follows from the above formulation that physical signals 
can be produced that closely approximate it. 

We noted that the magnitude of the impulse function is determined by the area under 
the limiting pulse. The actual shape of the limiting pulse is not really important, pro- 
vided that the area contained within it remains constant as its duration approaches zero. 
Physically, therefore, the unit impulse function at t= a may equally well be regarded 
as the pulse @,(t) of Figure 5.35 in the limiting case as T approaches zero. 

In some applications we need to consider a unit impulse function at time t = 0. This 
is denoted by 6(t) and is defined as the limiting case of the pulse @,(t) illustrated in 
Figure 5.36 as T approaches zero. It has the properties 


d(t)=0 (t#0) 


| 5(t) dt =1 


The sifting property 
An important property of the unit impulse function that is of practical significance is 


the so-called sifting property, which states that if f(t) is continuous at t = a then 


co 


| f(t)&(t - a) dt = f(a) (5.48) 


This is referred to as the sifting property because it provides a method of isolating, or 
sifting out, the value of a function at any particular point. 

For theoretical reasons it is convenient to use infinite limits in (5.48), while in reality 
finite limits can be substituted. This follows since for a < a < B, where a and B are 
constants, 


B 
| f(t) 6(t- a) dt = f(a) (5.49) 


a 
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For example, 


20 
| cos t6(t - i) dt = cos im =} 


5.5.10 Laplace transforms of impulse functions 


By the definition of the Laplace transform, we have for any a > 0 


co 


wot ar= | O(t-a)e™'dt 


0 


which, using the sifting property, gives the important result 

L{5(t- a)} =e (5.50) 
or, in terms of the inverse transform, 

£1 {fe} = 6(t— a) (5.51) 


As mentioned earlier, in many applications we may have an impulse function 6(f) at 
t = 0, and it is in order to handle such a function that we must carefully specify whether 
the lower limit in the Laplace integral defined in Section 5.2.1 is 0° or 0°. Adopting the 
notation 


co 


f(the dt 


0 


LALO} =| 


LAI} = | Se" dt 


we have 


o* ce 


esr | pines | f(the™ dt 


0 


If f(t) does not involve an impulse function at t= 0 then clearly LZ,{f()} = L{f}. 
However, if f(t) does involve an impulse function at t= 0 then 


| f(t) dt #0 


and it follows that 


LALO} # LALO} 
In Section 5.2.1 we adopted the definition 


LA f()} = LALO} 
so that (5.50) and (5.51) hold for a = 0, giving 


oy 


x1)=| O(the“dt=e" =1 


0 
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so that 
Poa = 1 (5.52) 


or, in inverse form, 


£11} = (0) (5.53) 


(2) This transform can be implemented in MATLAB using the sequence of commands 


Shans: Sy ic 
del=symi( {Danae (te), 
laplace (del) 


Likewise for (5.50); for example, if a = 2 then the Laplace transform of 6(t — 2) is 
generated by the commands 


del2=sym(‘Dirac(t-2)'’); 
laplace(del2) 


or directly using the command 
laplace(sym(‘Dirac(t-2)7’)) 


giving the answer exp (-2*s) in each case. 
In MAPLE the commands 


with(inttrans) : 
lajolace (Disc (e=2), i, SS) F 


return the answer e™”, 


Example 5.45 _— Determine z'| 2 } 
s+4 


Solution Since 


ss +4-4_,__ 4 


s+4 9 s'+4 s +4 
we have 
: 1 1 4 
cat pee'any-2'| oA 
s +4 s +4 
giving 


#3 | = )-2sina4 
s +4 
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2) In MATLAB this is obtained directly, with the commands 


ileplace(S*2/ (S224) )) e 
pretty (ans) 


generating the answer 
IDiLiere (iE) —2SiaZe 
The answers may also be obtained in MAPLE using the commands 


with(inttrans) : 
slahylieyoyleteren (3A) (pe) sip ie) G 


Example 5.46 _— Determine the solution of the differential equation 


dx dx = 
—+3=+2x=1+ &(t-4) (5.54) 
de = dt 


subject to the initial conditions x(0) = x(0) = 0. 


Solution Taking Laplace transforms in (5.54) gives 
[s?X(s) — sx(0) — X(0)] + 3[sX(s) — x(0)] + 2.X(s) = Lil} + L{d(t- 4)} 
which, on incorporating the given initial conditions and using (5.50), leads to 
(s? + 3s +.2)X(s) = : +e“ 
giving 


1 -4s 1 


MO Gey oe 


Resolving into partial fractions, we have 
1 1 2 -4s(_1 1 
X(s)=3(=+—=--—— |+ ( = } 
~ 1(; st+2 =) © \s+l st2 


which, on taking inverse transforms and using the result (5.45), gives the required 
response: 


x(t)= $1 +e%—-2e) +(e -e™™) Mt - 4) 
or, in an alternative form, 
r eae (0<t<4) 
x — 
Le(e'-l)e‘-(e'-le" (t= 4) 


We note that, although the response x(t) is continuous at t = 4, the consequence of the 
impulsive input at f= 4 is a step change in the derivative x(f). 
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L| 


5.0.41 


As was the case in Example 5.40, when considering Heaviside functions as forcing 
terms, it seems that the dsolve command in MATLAB cannot be used directly in 
this case. Using the maple command the following commands: 

nnayoulen( Toles cola. 112 (Se (ie )) , e221) eS) la ieie (ox (ie) pie) eee (ie) 

= dale (ig=4) 3” ) 

ans= 

GS ge Clie (sec) 7S" (it, 2) PPIoCiese (sx (ie) , ©) + 2ose (1) 

= ilwiiese (ie—4)) 

maple (‘dsolve({de,x(0)=0,D(x)(0)=0},x(t)), 

method=laplace; ’) 
output the required answer: 


x(t) =1/2-exp (-t)+1/2*exp(-2*t) -Heaviside(t-4) * 
exp (-2*t+8)+Heaviside(t-4) *exp(-t+4) 


Relationship between Heaviside step and 
impulse functions 


From the definitions of H(t) and 6(f), it can be argued that 
H(t) =| 6(t) dt (5.55) 


since the interval of integration contains zero if ¢ > 0 but not if t < 0. Conversely, 
(5.55) may be written as 


SOS . H(t) =H'(t) (5.56) 


which expresses the fact that H’(t) is zero everywhere except at t = 0, when the jump 
in H(t) occurs. 

While this argument may suffice in practice, since we are dealing with generalized 
functions a more formal proof requires the development of some properties of gener- 
alized functions. In particular, we need to define what is meant by saying that two 
generalized functions are equivalent. 

One method of approach is to use the concept of a test function @(t), which is a 
continuous function that has continuous derivatives of all orders and that is zero outside 
a finite interval. One class of testing function, adopted by R. R. Gabel and R. A. Roberts 
(Signals and Linear Systems, Wiley, New York, 1973), is 


-d' (a1) 
a(t) = 1& ({t] <d), where d=constant 
0 otherwise 


For a generalized function g(t) the integral 
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G(@) = | O(t)g(t) dt 
is evaluated. This integral assigns the number G(@) to each function @(t), so that G(@) 


is a generalization of the concept of a function: it is a linear functional on the space of 
test functions @(r). For example, if g(t) = 6(t) then 


G(@) =| 8(t) 0(t) dt = 0(0) 
so that in this particular case, for each weighting function @(t), the value 0(0) is 
assigned to G(@). 


We can now use the concept of a test function to define what is meant by saying that 
two generalized functions are equivalent or ‘equal’. 


Definition 5.2: The equivalence property 


If g,(t) and g,(t) are two generalized functions then g,(t) = g,(t) if and only if 


| A(t) gi(t) dt = | A(t) g(t) dt 
for all test functions 0(t) for which the integrals exist. 


The test function may be regarded as a ‘device’ for examining the generalized func- 
tion. Gabel and Roberts draw a rough parallel with the role of using the output of a 
measuring instrument to deduce properties about what is being measured. In such an 
analogy g(t) = g,(t) if the measuring instrument can detect no differences between 
them. 

Using the concept of a test function @(t), the Dirac delta function 6(t) may be 
defined in the generalized form 


| @(t) 6(t) dt = @(0) 


Interpreted as an ordinary integral, this has no meaning. The integral and the function 
6(t) are merely defined by the number 0(0). In this sense we can handle 6(f) as if it 
were an ordinary function, except that we never talk about the value of 6(t); rather we 
talk about the value of integrals involving 6(f). 
Using the equivalence property, we can now confirm the result (5.56), namely that 
d - 
(1) = ae) 


To prove this, we must show that 
| A(t) d(t) dt = | @(t)H’(t) dt (5.57) 


Integrating the right-hand side of (5.57) by parts, we have 
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Figure 5.37 
Piecewise-continuous 
function with jump 
discontinuities. 


oy 


| 0(t)H’(t) dt = [A(t) 0(t)]-.. | H(t) 0’(t) dt 


00 


=0- | 6’(t)dt (by the definitions of @(¢) and H(t)) 


= -[A(t)]> = (0) 


Since the left-hand side of (5.57) is also 6(0), the equivalence of 6(t) and H(t) is proved. 
Likewise, it can be shown that 


b= a)= ¢ H@—0)=H'(t-a) (5.58) 


The results (5.56) and (5.58) may be used to obtain the generalized derivatives of 
piecewise-continuous functions having jump discontinuities d,, d,, ..., d, at times 
ti, t,..., ¢, respectively, as illustrated in Figure 5.37. On expressing f(t) in terms of 
Heaviside step functions as in Section 5.5.1, and differentiating using the product rule, 
use of (5.56) and (5.58) leads to the result 


f= 2) +¥ 4,5(t- t;) (5.59) 


i=1 


where g’(t) denotes the ordinary derivative of f(t) where it exists. The result (5.59) tells 
us that the derivative of a piecewise-continuous function with jump discontinuities 
is the ordinary derivative where it exists plus the sum of delta functions at the discon- 
tinuities multiplied by the magnitudes of the respective jumps. 


By the magnitude d; of a jump in a function f(t) at a point ¢,, we mean the difference 
between the right-hand and left-hand limits of f(t) at ¢; that is, 


d; = f(t,+ 0)—fit, — 0) 


It follows that an upward jump, such as d, and d, in Figure 5.37, is positive, while a 
downward jump, such as d, in Figure 5.37, is negative. 

The result (5.59) gives an indication as to why the use of differentiators in practical 
systems is not encouraged, since the introduction of impulses means that derivatives 
increase noise levels in signal reception. In contrast, integrators have a smoothing effect 
on signals, and are widely used. 
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Example 5.47 — Obtain the generalized derivative of the piecewise-continuous function 


2r+1 (0<t<3) 
JO=si4 “(G3r<5) 
4 (t= 5) 


Figure 5.38 Piecewise- —_f(#) 
continuous function of 
Example 5.47. 


Solution f(t) is depicted graphically in Figure 5.38, and it has jump discontinuities of magni- 
tudes 1, —-12 and —5 at times ¢ = 0, 3 and 5 respectively. Using (5.59), the generalized 
derivative is 


f(t) =2(t) + 16(t) — 126(t — 3) — Sd(t - 5) 


where 


4t (0<t<3) 
g(t}=41 (3<t<5) 
0 (t=5) 


Example 5.48 A system is characterized by the differential equation model 


2 
dx soy Gy ay 43 (5.60) 
dt dt dt 
Determine the response of the system to a forcing function u(t) = e“ applied at time 


t= 0, given that it was initially in a quiescent state. 


Solution Since the system is initially in a quiescent state, the transformed equation correspond- 
ing to (5.60) is 


(s° + 5s + 6)X(s) = (35 + 1)U(s) 
giving 


X(s) = = +1 


s +5s +6 


U(s) 


In the particular case when u(t) = e“, U(s) = 1/(s + 1), so that 


De Lael) ee ee 22 a8 
(st1l)(s+2)(s+3) stl s+2 s+3 
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which, on taking inverse transforms, gives the desired response as 
x(t)=-e'+5e%-4e*" (t= 0) 
One might have been tempted to adopt a different approach and substitute for u(r) 


directly in (5.60) before taking Laplace transforms. This leads to 


5—+6x =e'-3e'=-2e" 


dx, dx 
dr dt 


which, on taking Laplace transforms, leads to 


2 
245 + 6)X - -—s 
(s Pee) stl 


giving 


-2 -1 2 1 
XxX Se 
Oe CoG ts) tl eae as 
which, on inversion, gives 
x(t)=-e'+2e%-e* (¢20) 


Clearly this approach results in a different solution, and therefore appears to lead to a 
paradox. However, this apparent paradox can be resolved by noting that the second 
approach is erroneous in that it ignores the important fact that we are dealing with 
causal functions. Strictly speaking, 


u(t) = e‘H(t) 


and, when determining duw/dt, the product rule of differential calculus should be 
employed, giving 


du_ =t ad 
= e‘H(t)+e rr 


=—e“H(t) + e“d(t) 


Substituting this into (5.60) and taking Laplace transforms gives 


2 1 1 3s+1 
+5s+6)X(s) = —~+3/-——4+]]= 
. poe stl ( s+] } stl 


That is, 


Xs) = oie 1 
(s+ 1)(s°'+5s+6) 


leading to the same response 
x(t)=—-e'+5e%-4e* (¢=0) 


as in the first approach above. 
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The differential equation used in Example 5.48 is of a form that occurs frequently in 
practice, so it is important that the causal nature of the forcing term be recognized. 

The derivative 6’(t) of the impulse function is also a generalized function, and, using 
the equivalence property, it is readily shown that 


co 


| A(t)6(t) dt = -f"(0) 


or, more generally, 


co 


| f(t) 6(t- a) dt = -f"(a) 


provided that f’(t) is continuous at f= a. 
Likewise, the nth derivative satisfies 


| f(t)&" (t -a) dt = (-1)"f(a) 


provided that f(r) is continuous at f= a. 
Using the definition of the Laplace transform, it follows that 


LS — a)} =s"e* 


and, in particular, 


LLSt)} = 5" (5.61) 


5.5.12 Exercises 


Check your answers using MATLAB or MAPLE whenever possible. 


2 
25 Obtain the inverse Laplace transforms of the (c) dx +7 Ot i 1a¢ = d(t - 3) 
following: t dt 
(a) __2s+1 (b) cial (c) st +2 subject to x = 1 and aa at f= 0 
(s+2)(s+3) °° 44 °° S425 45 a 


26 — Solve for t= 0 the following differential equations, 


subject to the specified initial conditions: 27 Obtain the generalized derivatives of the following 


piecewise-continuous functions: 


(a) Qe 7 OX 4 y= 24 H(- 2) ; 
dt dt 3r (0<t< 4) 
re (a) f(t)=)2t-3 (4<1t< 6) 
Spee ae 5 (t = 6) 
(b) 2% 464% 4 134 = ar- 2m) t (0<t<1) 
. - (b) g(t)=42-t (1 <t< 2) 
subject to x= 0 and ¥ =0 at r=0 0 (t = 2) 
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2t+5 (0<St< 2) 
(c) f(t)=49-34 (2 


28 Solve for ¢ = 0 the differential equation 


2. 
So+ 7S + r= 2043 =m 


dt 


subject to x = 0 and dx/dt = 2 at t= 0 and where 


u(t) =e “H(t). 


dx +@x=f(t) (t= 0) 
< 4) dt 


Show that 


zt x()= 4 H(t= nT) sin o(¢= nT) (t =0) 


dt 
and sketch the responses from t= 0 to t = 6/@ for 
the two cases (a) T= 1/@ and (b) T= 21/o. 


29 ~_—A periodic function f(t) is an infinite train of unit 30 = Animpulse voltage £6(t) is applied at time t = 0 
impulses at t= 0 and repeated at intervals of ¢ = T. to a circuit consisting of a resistor R, a capacitor 
Show that C and an inductor L connected in series. Prior to 


application of this voltage, both the charge on 


L{f(} = a the capacitor and the resulting current in the 
l-e circuit are zero. Determine the charge q(t) on the 
The response of a harmonic oscillator to such a periodic capacitor and the resulting current i(t) in the circuit 
stimulus is determined by the differential equation at time ¢. 
5.5.13 Bending of beams 


Figure 5.39 
Transverse deflection 
of a beam: (a) initial 
position; (b) displaced 
position. 


So far, we have considered examples in which Laplace transform methods have been 
used to solve initial-value-type problems. These methods may also be used to solve 
boundary-value problems, and, to illustrate, we consider in this section the application 
of Laplace transform methods to determine the transverse deflection of a uniform thin 
beam due to loading. 

Consider a thin uniform beam of length / and let y(x) be its transverse displacement, 
at distance x measured from one end, from the original position due to loading. The 
situation is illustrated in Figure 5.39, with the displacement measured upwards. Then, 
from the elementary theory of beams, we have 


4 


Erte = -W(x) (5.62) 
dx 


where W(x) is the transverse force per unit length, with a downwards force taken to be 
positive, and £7 is the flexural rigidity of the beam (£ is Young’s modulus of elasticity 
and J is the moment of inertia of the beam about its central axis). It is assumed that the 
beam has uniform elastic properties and a uniform cross-section over its length, so that 
both E and / are taken to be constants. 


y y 


W(x) 
(a) (b) 
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Equation (5.62) is sometimes written as 


4 
EIYY = Wx) 
dx 

where y(x) is the transverse displacement measured downwards and not upwards as 
in (5.62). 

In cases when the loading is uniform along the full length of the beam, that is 
W(x) = constant, (5.62) may be readily solved by the normal techniques of integral 
calculus. However, when the loading is non-uniform, the use of Laplace transform 
methods has a distinct advantage, since by making use of Heaviside unit functions and 
impulse functions, the problem of solving (5.62) independently for various sections of 
the beam may be avoided. 

Taking Laplace transforms throughout in (5.62) gives 


EI[s*¥(s) — s*y(0) — s*y,(0) — sy,(0) - (0) = -W) (5.63) 
where 
_ (dy _ (dy _(dy 
y,(0) te y(0) cre yx(0) Coe 


and may be interpreted physically as follows: 


Ely,(0) _ is the shear at x = 0 

Ely,(0) is the bending moment at x = 0 
y,(0O) is the slope at x = 0 
(0) is the deflection at x = 0 


Solving (5.63) for y(s) leads to 
15) =H), 2D, WO , BO WO eh 
s Ss 


Ss s Ss 
Thus four boundary conditions need to be found, and ideally they should be the shear, 
bending moment, slope and deflection at x = 0. However, in practice these boundary 
conditions are not often available. While some of them are known, other boundary con- 
ditions are specified at points along the beam other than at x = 0, for example conditions 
at the far end, x = /, or conditions at possible points of support along the beam. That is, 
we are faced with a boundary-value problem rather than an initial-value problem. 

To proceed, known conditions at x = 0 are inserted, while the other conditions among 
(0), v,(0), v(0) and y,(0) that are not specified are carried forward as undetermined 
constants. Inverse transforms are taken throughout in (5.45) to obtain the deflection 
y(x), and the outstanding undetermined constants are obtained using the boundary con- 
ditions specified at points along the beam other than at x = 0. 

The boundary conditions are usually embodied in physical conditions such as the 
following: 


(a) The beam is freely, or simply, supported at both ends, indicating that both the 
bending moments and deflection are zero at both ends, so that y = d’y/dx* = 0 at 
both x = 0 and x =/. 

(b) At both ends the beam is clamped, or built into a wall. Thus the beam is horizontal 
at both ends, so that y = dy/dx = 0 at both x = 0 andx =/. 
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Example 5.49 


Figure 5.40 
Loaded beam of 
Example 5.49. 


Solution 


(c) The beam is a cantilever with one end free (that is, fixed horizontally at one end, 
with the other end free). At the fixed end (say x = 0) 


po a5 atx =0 
dx 


and at the free end (x =/), since both the shearing force and bending moment are zero, 


2 3 
dy dy 9 atx=/ 
dx dx 


If the load is not uniform along the full length of the beam, use is made of Heaviside 
step functions and impulse functions in specifying W(x) in (5.62). For example, a 
uniform load w per unit length over the portion of the beam x = x, to x = x, is specified 
as WH(x — x,) — wH(x — x>), and a point load w at x = x, is specified as wd(x — x). 


Figure 5.40 illustrates a uniform beam of length /, freely supported at both ends, bending 
under uniformly distributed self-weight W and a concentrated point load P at x = $1. 
Determine the transverse deflection y(x) of the beam. 


As in Figure 5.39, the origin is taken at the left-hand end of the beam, and the deflection 
y(x) measured upwards from the horizontal at the level of the supports. The deflection 
y(x) is then given by (5.62), with the force function W(x) having contributions from the 
weight W, the concentrated load P and the support reactions R, and R,. However, since 
we are interested in solving (5.62) for 0 < x < /, point loads or reactions at the end 
x =I may be omitted from the force function. 

As a preliminary, we need to determine R,. This is done by taking static moments 
about the end x = /, assuming the weight W to be concentrated at the centroid x = +1, 
giving 

Rl = 5W1+ P3l 
or 
R, =3W+2P 


The force function W(x) may then be expressed as 
W(x) = 7 H(x) + P&(x- tN) - (LW + 2P)5(x) 
with a Laplace transform 


W(s) = aa Pe? -(1W+2P) 
S 
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Since the beam is freely supported at both ends, the deflection and bending moments 
are zero at both ends, so we take the boundary conditions as 


y=0 atx=Oandx=/ 


2 
dy_9 atx=Oandx=/ 


dx 
The transformed equation (5.64) becomes 
Ys) =-— Ae Fe? -wetpyh| -4@ , BO 
Bi Is° “ Ss Ss Ss 


Taking inverse transforms, making use of the second shift theorem (Theorem 5.4), 
gives the deflection y(x) as 


W 
y(x) = - mare P(x - LPH $1) LGW +P] 


+ y,(0)x + 4y3(0)x° 


To obtain the value of the undetermined constants y,(0) and y;(0), we employ the 
unused boundary conditions at x = /, namely y(/) = 0 and y,(/) = 0. For x > 4/ 


ve) =| Bate P(x- il) ‘Gre +y,(0)x +3 y3(0)x° 


d’y _1 | Wx =4 2P , 
f= n(2) = ala + P(x-11)- (+22) nos 


Thus taking y,(/) = 0 gives y,(0) = 0, and taking y(/) = 0 gives 


7c WI +% PI - Wl -{PI’)+ y(0)1 = 0 
so that 
¥,(0) = Fiaws 5 p) 
Substituting back, we find that the deflection )(x) is given by 


4 
x P 
y(x) = - FH -as+ 4Px)- Eg Px- be) - 2 (ety) 


or, for the two sections of the beam, 


-F(B-ge+ats)- 2g Px - tx’) (0<x <I) 
(x) = : 

W x 3 5 

-E(e-be +i Px ren e-iv1-LP) di<x<d 
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31 


32 


Input 


U xX 
7 


Figure 5.41 


5.5.14 Exercises 


Find the deflection of a beam simply supported at 
its ends x = 0 and x = /, bending under a uniformly 
distributed self-weight M and a concentrated load 
Watx= i l. 


A cantilever beam of negligible weight and of 
length / is clamped at the end x = 0. Determine the 
deflection of the beam when it is subjected to a load 


Ge =o Transfer functions 


5.6.1 Definitions 


SS 


per unit length, w, over the section x = x, to x =X. 
What is the maximum deflection ifx, = 0 and x, =/? 


A uniform cantilever beam of length / is subjected 
to a concentrated load W at a point distance b from 
the fixed end. Determine the deflection of the beam, 
distinguishing between the sections 0 < x < b and 
b<xSl. 


The transfer function of a linear time-invariant system is defined to be the ratio of 
the Laplace transform of the system output (or response function) to the Laplace trans- 
form of the system input (or forcing function), under the assumption that all the initial 
conditions are zero (that is, the system is initially in a quiescent state). 

Transfer functions are frequently used in engineering to characterize the input— 
output relationships of linear time-invariant systems, and play an important role in the 


analysis and design of such systems. 


Consider a linear time-invariant system characterized by the differential equation 


—1 
d’x d" x 
a, + Qy-1 
t’ a 


+...+a x = 5,, 


d” Uu 


m 
t 


ee rT (5.65) 


where n = m, the as and bs are constant coefficients, and x(f) is the system response or 
output to the input or forcing term u(t) applied at time ¢= 0. Taking Laplace transforms 
throughout in (5.65) will lead to the transformed equation. Since all the initial condi- 
tions are assumed to be zero, we see from (5.15) that, in order to obtain the transformed 
equation, we simply replace d/dt by s, giving 


(a,8" + G,8") +... + d))X(s) = (b,,8" +... + by )Us) 


where X(s) and U(s) denote the Laplace transforms of x(t) and u(t) respectively. 
The system transfer function G(s) is then defined to be 


(8) 0 ys in ct 
G(s) = 48 — 2m . 0 
8 Piss. t Ay 


U(s) 


(5.66) 


with (5.66) being referred to as the transfer function model of the system characterized 


System Output 
Writing 


P(s) = 6,5" +...4+ bo 


m 


Transfer function 


block diagram. 


O(s) =a,8" +... 4+ a 


by the differential equation model (5.65). Diagramatically this may be represented by the 
input—output block diagram of Figure 5.41. 
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the transfer function may be expressed as 


es) = 2 
Os) 


where, in order to make the system physically realizable, the degrees m and n of the 
polynomials P(s) and Q(s) must be such that n = m. This is because it follows from 
(5.61) that if m > n then the system response x(f) to a realistic input u(t) will involve 
impulses. 

The equation Q(s) = 0 is called the characteristic equation of the system; its order 
determines the order of the system, and its roots are referred to as the poles of the 
transfer function. Likewise, the roots of P(s) = 0 are referred to as the zeros of the 
transfer function. 

It is important to realize that, in general, a transfer function is only used to character- 
ize a linear time-invariant system. It is a property of the system itself, and is independent 
of both system input and output. 

Although the transfer function characterizes the dynamics of the system, it provides 
no information concerning the actual physical structure of the system, and in fact sys- 
tems that are physically different may have identical transfer functions; for example, 
the mass—spring—damper system of Figure 5.12 and the LCR circuit of Figure 5.8 both 
have the transfer function 


yc. TE _ 
Us) ast+Bs+y 


In the mass—spring—damper system X(s) determines the displacement x(¢) of the mass 
and U(s) represents the applied force F(t), while @ denotes the mass, 8 the damping 
coefficient and y the spring constant. On the other hand, in the LCR circuit X(s) deter- 
mines the charge g(f) on the condenser and U(s) represents the applied emf e(f), while 
a denotes the inductance, f the resistance and y the reciprocal of the capacitance. 

In practice, an overall system may be made up of a number of components each 
characterized by its own transfer function and related operation box. The overall system 
input—output transfer function is then obtained by the rules of block diagram algebra. 

Since G(s) may be written as 


b,, (8 — Z,)(8 — 22)... (8 — Zn) 
G — —” UUW ems 
8) = 1, (8 PO —Ps) «(= Pw) 


where the z,s and p;s are the transfer function zeros and poles respectively, we observe 
that G(s) is known, apart from a constant factor, if the positions of all the poles and 
zeros are known. Consequently, a plot of the poles and zeros of G(s) is often used as 
an aid in the graphical analysis of the transfer function (a common convention is to 
mark the position of a zero by a circle O and that of a pole by a cross Xx). Since the 
coefficients of the polynomials P(s) and Q(s) are real, all complex roots always occur in 
complex conjugate pairs, so that the pole—zero plot is symmetrical about the real axis. 


The response x(f) of a system to a forcing function u(f) is determined by the differential 
equation 
2 
gf ky ip pays 5 
dt dt dt 


+3u 
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Solution 


Figure 5.42 
Pole (x)—zero (O) plot 
for Example 5.50. 


(a) 
(b) 


(c) 


(a) 


(b) 


(c) 


Determine the transfer function characterizing the system. 

Write down the characteristic equation of the system. What is the order of the 
system? 

Determine the transfer function poles and zeros, and illustrate them diagram- 
matically in the s plane. 


Assuming all the initial conditions to be zero, taking Laplace transforms throughout 
in the differential equation 


2. 
9624 129% 4 13x = 2 
dt dt dt 


+3u 


leads to 
(9s? + 12s + 13)X(s) = (28 + 3)U(s) 
so that the system transfer function is given by 
X(s 2s+3 
a ) ~ Os?-+ 128+ 13 
The characteristic equation of the system is 
9s? + 12s +13 =0 
and the system is of order 2. 
The transfer function poles are the roots of the characteristic equation 


9s’ + 125+ 13=0 


which are 
esis ¥(144 - 468) _ —2 +53 


18 3 


That is, the transfer function has simple poles at 
s=-24+j and s=-2-j 


The transfer function zeros are determined by equating the numerator polynomial 
2s + 3 to zero, giving a single zero at 


The corresponding pole—zero plot in the s plane is shown in Figure 5.42. 


Im (s} 
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A transfer function (tf) is implemented within MATLAB using the commands 


GS ise s*)) 
G = G(s) 


Thus, entering G=(2*s+3) /(9*s*2+12*s+13) generates 


2S +3 


transfer function = - 
Sis dh IAS a IL} 


The command poly (G) generates the characteristic polynomial, whilst the commands 
pole(G) and zero(G) generate the poles and zeros respectively. The command 
pzmap(G) draws the pole—zero map. 


Stability 


The stability of a system is a property of vital importance to engineers. Intuitively, we 
may regard a stable system as one that will remain at rest unless it is excited by an 
external source, and will return to rest if all such external influences are removed. Thus 
a stable system is one whose response, in the absence of an input, will approach zero 
as time approaches infinity. This then ensures that any bounded input produces a 
bounded output; this property is frequently taken to be the definition of a stable linear 
system. 

Clearly, stability is a property of the system itself, and does not depend on the 
system input or forcing function. Since a system may be characterized in the s domain 
by its transfer function G(s), it should be possible to use the transfer function to specify 
conditions for the system to be stable. 

In considering the time response of 


X)=GOUQ;, c= 22 
Os) 


to any given input u(A), it is necessary to factorize the denominator polynomial 
O(s)=a,8" +a, 8"! +...4+ dp 


and various forms of factors can be involved. 


Simple factor of the form s + a, with a@ real 


This corresponds to a simple pole at s = —a, and will in the partial-fractions expansion 
of G(s) lead to a term of the form c/(s + @) having corresponding time response 
ce H(t), using the strict form of the inverse given in (5.12). If @ > 0, so that the pole 
is in the left half of the s plane, the time response will tend to zero as t > ~. Ifa < 0, 
so that the pole is in the right half of the s plane, the time response will increase without 
bound as t > o. It follows that a stable system must have real-valued simple poles of 
G(s) in the left half of the s plane. 

a = 0 corresponds to a simple pole at the origin, having a corresponding time 
response that is a step cH(t). A system having such a pole is said to be marginally 
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stable; this does not ensure that a bounded input will lead to a bounded output, since, 
for example, if such a system has an input that is a step d applied at time t = 0 then the 
response will be a ramp cdtH(t), which is unbounded as t > ~, 


Repeated simple factors of the form (s + a)", with @ real 


This corresponds to a multiple pole at s = —a, and will lead in the partial-fractions 
expansion of G(s) to a term of the form c/(s + a)" having corresponding time response 
[c/(n — 1)!]t” ee“ H(t). Again the response will decay to zero as f > ~ only if a> 0, 
indicating that a stable system must have all real-valued repeated poles of G(s) in the 
left half of the s plane. 


Quadratic factors of the form (s + a)’ + B’, with a and B real 


This corresponds to a pair of complex conjugate poles at s = —a + jB, s=—a— jf, and 
will lead in the partial-fractions expansion of G(s) to a term of the form 


c(st+ a@)+dB 
(st ay+pP 


having corresponding time response 
e“(ccos Bt + dsin Bt) =Ae™ sin(Bt+ y) 


where A = \(c? + d’) and y= tan ‘(c/d). 

Again we see that poles in the left half of the s plane (corresponding to a@ > 0) have 
corresponding time responses that die away, in the form of an exponentially damped 
sinusoid, as t > oo, A stable system must therefore have complex conjugate poles 
located in the left half of the s plane; that is, all complex poles must have a negative 
real part. 

If ~@= 0, the corresponding time response will be a periodic sinusoid, which will not 
die away as ¢ > oe. Again this corresponds to a marginally stable system, and will, for 
example, give rise to a response that increases without bound as t — co when the input 
is a sinusoid at the same frequency . 

A summary of the responses corresponding to the various types of poles is given in 
Figure 5.43. 

The concept of stability may be expressed in the form of Definition 5.3. 


Definition 5.3 


A physically realizable causal time-invariant linear system with transfer function 
G(s) is stable provided that all the poles of G(s) are in the left half of the s plane. 


The requirement in the definition that the system be physically realizable, that is n = m 
in the transfer function G(s) of (5.66), avoids terms of the form s’”” in the partial- 
fractions expansion of G(s). Such a term would correspond to differentiation of degree 
m —n, and were an input such as sin wt used to excite the system then the response 
would include a term such as w”™” sin wt or w”"" cos wt, which could be made as large 
as desired by increasing the input frequency a. 
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Figure 5.43 Poles of G(s) in Poles in complex Corresponding Nature of response 
Relationship between form ¢ + ja s plane time response 
transfer function poles 


and time response. a 
g=w=0 Constant 
g t 
wo 
g=w=0 
(multiplicity 2) a Ramp 
t 
wo 
fet west ee 
e lecay 
t 
@ 
o>0.0=0 Exponential 
— - : growth 
w 
a=0,w>0 Sinusoidal 
Go t 
w 
_ Linearly 
ie 0, ad 0 growing 
(multiplicity 2) a sinusoidal 


Exponentially 


a<0,@>0 decaying 
sinusoidal 
Exponentially 

a>0,w>0 growing 
sinusoidal 


In terms of the poles of the transfer function G(s), its abscissa of convergence o, 
corresponds to the real part of the pole located furthest to the right in the s plane. For 
example, if 


stl 


9) = eas) 


then the abscissa of convergence o, = —2. 
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Example 5.51 


Solution 


It follows from Definition 5.3 that the transfer function G(s) of a stable system has an 
abscissa of convergence o, = —a, with a > 0. Thus its region of convergence includes 
the imaginary axis, so that G(s) exists when s = j@. We shall return to this result when 
considering the relationship between Laplace and Fourier transforms in Section 8.4.1. 

According to Definition 5.3, in order to prove stability, we need to show that all the 
roots of the characteristic equation 


O(s) =a,s8" + 4,)s"'+...+a,8 + a,=0 (5.67) 


have negative real parts (that is, they lie in the left half of the s plane). Various criteria 
exist to show that all the roots satisfy this requirement, and it is not necessary to solve 
the equation to prove stability. One widely used criterion is the Routh-Hurwitz criterion, 
which can be stated as follows: 


A necessary and sufficient condition for all the roots of equation (5.67) 
to have negative real parts is that the determinants A,, A,,..., A, are 
all positive, where 


Dra an 0 0 see 0 
(Oh a) Qn-2 Qral an Borns 0 
5.68 
AN, = ays Qy-4 a)-3 Qy-2 shes) 0 ( ) 
Qy-(2r-1) Qy-2r Qy—2y-1 Qy-27-2 IND ay-; 


it being understood that in each determinant all the as with subscripts 
that are either negative or greater than n are to be replaced by zero. 


Show that the roots of the characteristic equation 
si + 9s? + 3357 + 515+ 26=0 


all have negative real parts. 


In this case n = 4, ay = 26, a, = 51, a, = 33, a; = 9, ag = 1 and a, = 0 (r > 4). The 
determinants of the Routh—Hurwitz criterion are 


A, |a,-1| |as| |9| 9 > 0 


hy as Qy-1 a, = a3 a4 
Qn-3 An-2 a, a) 
9 1 
= = 246 > 0 
5A, 33 
Qn-| ay 0 a3 a4 0 
As = |ay-3 Gnz Gp-i a, a, a3 
Ans An-4 Ay-3 a; a a 
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An: Ay 0 0 a, ay 0 0 

Ae Qn-3 An-2 Ay -1 a, _ ay a a3; a4 
GQn-5 An-4— An-3 An-2 a, do a, ay 
Gn-7 An-6 Ans An-4 a3 A yz GA \, a 
9 1 0 0 
1 1 

= > 23 . = 26A; > 0 

0 26 51 37 
0 0 0 26 


Thus A, > 0, A, > 0, A; > 0 and A, > 0, so that all the roots of the given characteristic 
equation have negative real parts. This is readily checked, since the roots are —2, —1, 
—3 + j2 and —3 — j2. 


The steady motion of a steam-engine governor is modelled by the differential equations 
mij + bi + dn- ea=0 (5.69) 
1a =-fn (5.70) 


where 77 is a small fluctuation in the angle of inclination, @ a small fluctuation in the 
angular velocity of rotation, and m, b, d, e, fand J, are all positive constants. Show that 
the motion of the governor is stable provided that 


bd . ef 


m Ih 


Differentiating (5.69) gives 
mij + bij + dy -— ea =0 
which, on using (5.70), leads to 


mij + bt + dr + Ly =0 
0 
for which the corresponding characteristic equation is 
ms? + bs’ + ds + ef’ =o 
0 


This is a cubic polynomial, so the parameters of (5.67) are 


n=3, ay= 2 a,=d, a=b, a,y=m (a,=0,r> 3) 
0 


The determinants (5.68) of the Routh—Hurwitz criterion are 


A, =|a|=b>0 


Ae Ole b m)_,,_mef 
. a ay ef/Il, d Ih 
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5.6.3 


(and so A, > 0 provided that bd — mef/I, > 0 or bd/m > ef/l), and 


a a, 0 
A;=|d) @; Ql=aA,>0 ifA,>0 


0 0 ao 
Thus the action of the governor is stable provided that A, > 0; that is, 
bd. ef 
m Ih 


Impulse response 


From (5.66), we find that for a system having transfer function G(s) the response x(¢) 
of the system, initially in a quiescent state, to an input u(t) is determined by the 
transformed relationship 


X(s) = G(s) Us) 


If the input u(t) is taken to be the unit impulse function 6(f) then the system response 
will be determined by 


X(s) = G(s) L{ S(O} = G(s) 


Taking inverse Laplace transforms leads to the corresponding time response A(t), which 
is called the impulse response of the system (it is also sometimes referred to as the 
weighting function of the system); that is, the impulse response is given by 


h(t) = £'{X(s)} = LG(s)} (5.71) 


We therefore have the following definition. 


Definition 5.4: Impulse response 


The impulse response A(t) of a linear time-invariant system is the response of the 
system to a unit impulse applied at time t= 0 when all the initial conditions are zero. 
It is such that L{h(t)} = G(s), where G(s) is the system transfer function. 


Since the impulse response is the inverse Laplace transform of the transfer function, 
it follows that both the impulse response and the transfer function carry the same informa- 
tion about the dynamics of a linear time-invariant system. Theoretically, therefore, it is 
possible to determine the complete information about the system by exciting it with an 
impulse and measuring the response. For this reason, it is common practice in engineering 
to regard the transfer function as being the Laplace transform of the impulse response, 
since this places greater emphasis on the parameters of the system when considering 
system design. 

We saw in Section 5.6.2 that, since the transfer function G(s) completely characterizes 
a linear time-invariant system, it can be used to specify conditions for system stability, 
which are that all the poles of G(s) lie in the left half of the s plane. Alternatively, 
characterizing the system by its impulse response, we can say that the system is stable 
provided that its impulse response decays to zero as t > ©, 
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Solution 


5.6.4 


Theorem 5.6 
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Determine the impulse response of the linear system whose response x(f) to an input 
u(t) is determined by the differential equation 


dx, .dx _ 
— +5 + 6x = 5u(t) (5.72) 
dt dt 


The impulse response /(f) is the system response to u(t) = d(t) when all the initial 
conditions are zero. It is therefore determined as the solution of the differential equation 


dh, fh. 6p = 554) (5.73) 
dt dt 

subject to the initial conditions A(0) = h(0) = 0. Taking Laplace transforms in (5.73) gives 
(s° + 5s + 6)H(s) = 5L{6(t)} =5 

so that 


H(s) 5 5 5 


~ (s+3)(s+2) +2 543 
which, on inversion, gives the desired impulse response 
h(t) = 5(e* - e* 
Alternatively, the transfer function G(s) of the system determined by (5.72) is 
a 
sv +55+6 
so that h(t) = £'{G(s)} = 5(e— e~*) as before. 


G(s) = 


Note: This example serves to illustrate the necessity for incorporating 0" as the lower 
limit in the Laplace transform integral, in order to accommodate for an impulse applied 
at t= 0. The effect of the impulse is to cause a step change in x(f) at f= 0, with the initial 
condition accounting for what happens up to 0°. 


In MATLAB a plot of the impulse response is obtained using the commands 
sei (7 s7 ) 
G=G(s) 
impulse (G) 


Initial- and final-value theorems 


The initial- and final-value theorems are two useful theorems that enable us to predict 
system behaviour as t > 0 and t > © without actually inverting Laplace transforms. 


The initial-value theorem 


If f(t) and f(t) are both Laplace-transformable and if lim sF(s) exists then 


lim f() = f(0") = lim sF(s) 
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Proof From (5.13), 


co 


Lif} = | SO edt = sF(s) —fO) 


where we have highlighted the fact that the lower limit is 0°. Hence 


co 


lim [sF(s) —f(0)] = lim | fhe“ dt 


ot co 
= lim | fe dt + lim | fe“ dt (5.74) 
Se o- S30 ot 


If f(t) is discontinuous at the origin, so that f(0*) # (0), then, from (5.59), f() contains 
an impulse term [ f(0*) — f(0-)]6(d, so that 


im | fe" dt =f") -f(0) 


Also, since the Laplace transform of f(t) exists, it is of exponential order and we have 
lim | SMe" dt=0 
Soo ot 


so that (5.74) becomes 
lim sF(s) — (0) = 10") ~ 0) 
giving the required result: 
lim sF(s) = f(0") 
If f(t) is continuous at the origin then f(t) does not contain an impulse term, and the 
right-hand side of (5.74) is zero, giving 
lim sF(s) = f(0) = f(0") 


It is important to recognize that the initial-value theorem does not give the initial 
value f(0-) used when determining the Laplace transform, but rather gives the value of 
f(é as t > 0°. This distinction is highlighted in the following example. 


Example 5.54 = The circuit of Figure 5.44 consists of a resistance R and a capacitance C connected in 
series together with constant voltage source E. Prior to closing the switch at time ¢= 0, 
both the charge on the capacitor and the resulting current in the circuit are zero. Deter- 
mine the current i(f) in the circuit at time f after the switch is closed, and investigate the 
use of the initial-value theorem. 
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Solution 
t=0 R 
E “Y 1. 
it) 
Figure 5.44 


RC circuit of 
Example 5.54. 


Theorem 5.7 


Proof 


5.6 TRANSFER FUNCTIONS 


Applying Kirchhoff’s law to the circuit of Figure 5.44, we have 


nik [iar= a, 
C 


which, on taking Laplace transforms, gives the transformed equation 


aie ito 
GC Ss AY 


Therefore 


E,/R 
Ks) = —e 
(3) = Re 


Taking inverse transforms gives the current i(f) at t = 0 as 


. E, -t/RC 
t =—e 
i( ) R 


Applying the initial-value theorem, 


oes _ sE/R —.... ER _ Eo 
#) = limsi(s) = lim =“ = jim ——2— = 22 
a y= i RC ieee 
That is, 
E 
(0*) = 2 
i(0") R 
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(5.75) 


a result that is readily confirmed by allowing t > 0° in (5.75). We note that this is 
not the same as the initial state i(0) = 0 owing to the fact that there is a step change in 


i(t) att=0. 


The final-value theorem 


If f(t) and f(t) are both Laplace-transformable and lim f(t) exists then 
too 


lim f(‘) = lim sF(s) 
too s>0 


From (5.13), 


Lf} = | _fOe"dt= ss) -fO) 
Taking limits, we have 


oy oy 


lim [sF(s) — f(0)] = lim | 


O~ O~ 


= lim A) ~ £0) 
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Example 5.55 


Solution 


giving the required result: 
lim f(‘) = lim sF(s) 
t—oo s-> 
end of theorem 


The restriction that lim f(t) must exist means that the theorem does not hold for func- 


tions such as e’, which tends to infinity as t > o, or sin wt, whose limit is undefined. 
Since in practice the final-value theorem is used to obtain the behaviour of f(t) as t > 
from knowledge of the transform F(s), it is more common to express the restriction in 
terms of restrictions on F(s), which are that sF(s) must have all its poles in the left half 
of the s plane; that is, sF(s) must represent a stable transfer function. It is important that 
the theorem be used with caution and that this restriction be fully recognized, since the 
existence of lim sF(s) does not imply that f(t) has a limiting value as t > ©. 
sa 


Investigate the application of the final-value theorem to the transfer function 


1 


PT Ga 


(5.76) 


lim sF(s) = lim ———*—-. = 0 
ee a 


so the use of the final-value theorem implies that for the time function f(¢) corresponding 
to F(s) we have 

lim f() = 0 

too 


However, taking inverse transforms in (5.76) gives 
fl) = He" 0) 


implying that /(¢) tends to infinity as t > oe. This implied contradiction arises since the 
theorem is not valid in this case. Although lim sF(s) exists, sF(s) has a pole at s = 3, 
which is not in the left half of the s plane. *~ 


The final-value theorem provides a useful vehicle for determining a system’s steady- 
state gain (SSG) and the steady-state errors, or offsets, in feedback control systems, 
both of which are important features in control system design. 

The SSG of a stable system is the system’s steady-state response, that is the response 
as t > ce, to a unit step input. For a system with transfer function G(s) we have, from 
(5.66), that its response x(f) is related to the input u(t) by the transformed equation 


X(s) = G(s) Us) 


For a unit step input 


u(t)=1H() giving U(s)= + 
Ss 
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Solution 


R(s) E(s) X(s) 
Ore 


Figure 5.45 Unity 
feedback control 
system. 


Example 5.57 


Solution 
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so that 


From the final-value theorem, the steady-state gain is 


SSG = lim x(t) = lim 5X(s) = lim G(s) 
too s- so 


Determine the steady-state gain of a system having transfer function 


G(s) = 20+ 3s) 


s°+75+10 


The response x(f) to a unit step input u(t) = 1H(¢) 1s given by the transformed equation 

X(s) = G(s)U(s) = caer 1 

7+7s+108 

Then, by the final-value ee the steady-state gain is given by 

SSG = lim x(¢) = lim sX(s) = lim ey 

$0 67 +754 10 
Note that for a step input of magnitude K, that is u(t) = KH(0), the steady-state response 
will be lim &G(s) = 2K; that is, 
s30 


steady-state response to step input = SSG x magnitude of step input 


A unity feedback control system having forward-path transfer function G(s), reference 
input or desired output r(f) and actual output x(f) is illustrated by the block diagram 
of Figure 5.45. Defining the error to be e(¢) = r(4) — x(d), it follows that 


G(s)E(s) = X(s) = R(s) — E(s) 


giving 
E(s) = Rs) | 
1+ G(s) 
Thus, from the final-value theorem, the steady-state error (SSE) is 
SSE = lim e(#) = lim sE(s) = lim pee (5.77) 
0 1+ G(s) 


Determine the SSE for the system of Figure 5.45 when G(s) is the same as in 
Example 5.50 and r(f) is a step of magnitude K. 


Since r(t) = KH(t), we have R(s) = K/s, so, using (5.77), 


‘K/s _ Kk 
SSE = lim — = 
301+G(s)_ 1+SSG 
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34 


55 


36 


SY 


where SSG = 2 as determined in Example 5.56. Thus 


It is clear from Example 5.57 that if we are to reduce the SSE, which is clearly 
desirable in practice, then the SSG needs to be increased. However, such an increase 
could lead to an undesirable transient response, and in system design a balance must be 
achieved. Detailed design techniques for alleviating such problems are not considered 
here; for such a discussion the reader is referred to specialist texts (see for example 
J. Schwarzenbach and K. F. Gill, System Modelling and Control, Edward Arnold, 


London, 1984). 


5.6.5 Exercises 


The response x(t) of a system to a forcing function 
u(t) is determined by the differential equation model 
2 
Or 98k 4 5p lt a oy 
dr dt dt 
(a) Determine the transfer function characterizing 
the system. 
(b) Write down the characteristic equation of the 
system. What is the order of the system? 
(c) Determine the transfer function poles and 
zeros, and illustrate them diagrammatically in 
the s plane. 


Repeat Exercise 34 for a system whose response 
x(t) to an input u(¢) is determined by the differential 
equation 


dx .d°x dx du 


- du 
CX 4 52841724 13x = SH 


dt dt dt dt dt 


+6 


Which of the following transfer functions represent 
stable systems and which represent unstable systems? 


(a) s-1 (b (s+2)(s - 2) 
(s+2)(s° +4) (s+ 1)(s- 1)(s +4) 

(c) ss 6 
(s+2)(s +4) (s° +s +1)(s+ 1 


(e) 5(s +10) 
(s+ 5)(s°-s +10) 


Which of the following characteristic equations are 
representative of stable systems? 

(a) s°—4s+13=0 

(b) 5s° + 13s? + 31s + 15=0 


33 


Sg) 


40 


(c) s+ +s4+1=0 
(d) 24s*+ 11s? + 26s? + 455 + 36 =0 
(e) 6 +2s°+2s+1=0 


The differential equation governing the motion of a 
mass-—spring—damper system with controller is 
a: 2 
mX 4 cL 4 KES Ky =0 
dt dt dt 
where m, c, K and r are positive constants. Show 
that the motion of the system is stable provided that 
r<c/m. 


The behaviour of a system having a gain controller 
is characterized by the characteristic equation 
si +2 + (K+ 2)s°+75+K=0 


where K is the controller gain. Show that the system 
is stable provided that K > 2.1. 


A feedback control system has characteristic equation 
si + 15Ks* + (2K - 1)s+5K=0 


where K is a constant gain factor. Determine the 
range of positive values of K for which the system 
will be stable. 


Determine the impulse responses of the linear 
systems whose response x(t) to an input u(f) is 
determined by the following differential equations: 


2 
(a) $84 158% 4 56x = 3u(2) 
dt 


dt 
2 

(bo) $8482 425% = ws) 
dt dt 


www.2Ofile.org 


42 


43 


44 
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(a) 1+3e‘sin2t¢ (b) t?e* 


2 
(c) $¥- 298 - gy = u(y) 
dt dt (c) 3-2e* + e“‘cos2r 
dx dx 
(d) a2 ~ or + 13x = u(t) 45 Using the final-value theorem, check the value 
: é obtained for 7,(t) as t > ° for the circuit of 
What can be said about the stability of each of the Example 5.28. 
systems? 
46 Discuss the applicability of the final-value theorem 
The response of a given system to a unit step for obtaining the value of i,(f) as t > © for the 
u(t) = 1H(t) is given by circuit of Example 5.29. 


-t =2 
x(t)=1-fe't+e 


What is the transfer function of the system? 


t 41-4 

-te 
6 47 — Use the initial- and final-value theorems to find the 
jump at ¢= 0 and the limiting value as t > ce for the 


solution of the initial-value problem 


Verify the initial-value theorem for the functions 


d 31 
(a) 2—3cost (b) Gt-1—(c) ¢ +. 3 sin 2¢ TS + Sy=4 40% + 25(0 
Verify the final-value theorem for the functions with (0°) =—-1. 
5.6.6 Convolution 


Convolution is a useful concept that has many applications in various fields of 
engineering. In Section 5.6.7 we shall use it to obtain the response of a linear system to 
any input in terms of the impulse response. 


Definition 5.5: Convolution 


Given two piecewise-continuous functions f(t) and g(t), the convolution of f(f) and 
g(t), denoted by f « g(A), is defined as 


f*E0= | S(Dgtt — 1) dt 


In the particular case when f(f) and g(f) are causal functions 
f@=at)=0 (7< 0), g-D=0 (t> 2) 


and we have 


t 


fxgQ= | S(O)g(t — 1) dt (5.78) 


The notation / « g(t) indicates that the convolution f « g is a function of ¢; that is, it could 
also be written as ( f * g)(f). The integral [*. f(t) g(t — T) dt is called the convolution 
integral. Alternative names are the superposition integral, Duhamel integral, folding 
integral and faltung integral. 

Convolution can be considered as a generalized function, and as such it has many of 
the properties of multiplication. In particular, the commutative law is satisfied, so that 


Seg =ge%f 
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or, for causal functions, 


| f@eg(t- t)dt= | f(t — T)g(t) dt (5.79) 


0 0 


This means that the convolution can be evaluated by time-shifting either of the two 
functions. The result (5.79) is readily proved, since by making the substitution T, = t — T 
in (5.78) we obtain 


f*xgh= | S(t — T)g(t)(-dT,) -| S(t- Tg(t) dt, = g* flO) 


0 


Example 5.58 For the two causal functions 
SO=tHO),  — g(t) = sin2¢ H(t) 
show that f « g(t) = g* f(t). 


t 


Solution | fet ne= | Tsin 2(t — T) dt 


0 0 
Integrating by parts gives 


f* a(t) =[} tTcos 2(t— Tt) + § sin2(t— 7)]}= 5 t—} sin 2¢ 
t 


| fu~ne(oar= | (t — T) sin2tdt 


0 0 
=[-}(t- T)cos2t— } sin2t}, = $¢— § sin2r 


so that f« g(t) = g* f(t). 


The importance of convolution in Laplace transform work is that it enables us to 
obtain the inverse transform of the product of two transforms. The necessary result for 
doing this is contained in the following theorem. 


Theorem 5.8 Convolution theorem for Laplace transforms 


If f(t) and g(t) are of exponential order 0, piecewise-continuous on ¢ = 0 and 
have Laplace transforms F(s) and G(s) respectively, then, for s > o, 


z| | fig(t- 7) a} = Lt fxg(t)t = F(s)G(s) 


or, in the more useful inverse form, 


L'{F(s)G(s)} =f * g(t) (5.80) 
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Figure 5.46 
Regions of integration. 


Example 5.59 


Solution 


5.6 TRANSFER FUNCTIONS 445 
By definition, 


F(s)G(s) = LL A tLig(ot = | e fix) a | egy) | 


0 0 


where we have used the ‘dummy’ variables x and y, rather than ¢, in the integrals to 
avoid confusion. This may now be expressed in the form of the double integral 


F(s)G(s) | | ee f(x) g(y) dx dy = | | e*™ F(x) g(y) dx dy 


R 
where R is the first quadrant in the (x, y) plane, as shown in Figure 5.46(a). On making 
the substitution 
x+y=t, y=t 


the double integral is transformed into 


F(s)G(s) = | | eo f(t— T)e(t)dtdt 


Ry 


where R, is the semi-infinite region in the (7, ¢) plane bounded by the lines t= 0 and 
T= 1, as Shown in Figure 5.46(b). This may be written as 


F(s)G(s) = | of | flt- Hated ar= | e“[g «f(] dt = L{g *f(O} 


0 


(a) Region R (b) Region Ry 


and, since convolution is commutative, we may write this as 
F(s)G(s) = LU f * g(t} 
which concludes the proof. 


end of theorem 


Using the convolution theorem, determine 4" \actsah- 
2 2 
s(s+2) 


We express 1/s*(s + 2) as (1/s”)[1/(s + 2)’]; then, since 
1 1 


S{th= : 
(s +2) 


,  Lte = 


i) 
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taking f(t) = t and g(f) = te in the convolution theorem gives 


et : | -| fu-nateyae= | (t—t)te*" dt 
Ss (s+2) r i 


which on integration by parts gives 


gg 13 | | =[-te[(r- tyr Hr- 27) - y= tft 14+ (t+ De 
s (s+2) 
We can check this result by first expressing the given transform in partial-fractions 
form and then inverting to give 
1 at ot 1 1 
== tide i a eee rarer 
s(s+2) Ss g+2 (s+2) 


so that 


v| | =—teitete tire = 1[t-14+(t+le"] 
s'(s +2) 


as before. 


5.6.7 System response to an arbitrary input 


The impulse response of a linear time-invariant system is particularly useful in practice 
in that it enables us to obtain the response of the system to an arbitrary input using the 
convolution integral. This provides engineers with a powerful approach to the analysis 
of dynamical systems. 

Let us consider a linear system characterized by its impulse response A(t). Then 
we wish to determine the response x(f) of the system to an arbitrary input u(t) such as 
that illustrated in Figure 5.47(a). We first approximate the continuous function u(t) by 
an infinite sequence of impulses of magnitude u(mAT), n = 0, 1, 2, ..., as shown in 
Figure 5.47(b). This approximation for u(t) may be written as 


u(t) = Y u(nAT)5(t~ nAT) AT (5.81) 


n=0 


Figure 5.47 u(t) 
Approximation to a 
continuous input. 


2AT 
(a) (b) 


Since the system is linear, the principle of superposition holds, so that the response of 
the system to the sum of the impulses is equal to the sum of the responses of the system 
to each of the impulses acting separately. Depicting the impulse response h(t) of the 
linear system by Figure 5.48, the responses due to the individual impulses forming the 
sum in (5.81) are illustrated in the sequence of plots in Figure 5.49. 
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Figure 5.48 


dt) 
Impulse response aid Linear dick 
of a linear system. system 
i 
e) ! 


A(t) 


oO t 
Output 
Input 
i u(O)ATO(t) ATu (O)h (1) 
«oar —-| 
6) t oO ; 
Output 
Input u(AT)AT ATu(AT) 
xd(t- AT) xh(t— AT) 
u{AT)AT [is | 
oO AT t ie) AT ‘ 
Output 
Input uQQATNAT ATu (2AT) 
xd(t— 2AT) xh(t — 2AT) 
u(2AT)AT | us | 
. ad ' 0 2AT t 
Output 
Input u(nAT)AT ATu(nAT) 
xd(t—nAT) xh(t—nAT) 
u(nAt) AT 
OG naT 1 O ve 


Figure 5.49 Responses due to individual impulses. 


Summing the individual responses, we find that the response due to the sum of the 
impulses is 


py u(nAT )h(t — nAT) AT (5.82) 
n=0 
Allowing AT > 0, so that nAT approaches a continuous variable 7, the above sum will 
approach an integral that will be representative of the system response x(f) to the 
continuous input u(t). Thus 
co t 
x(t) = | u(t)h(t — T) dt = | u(t)h(t— tT) dt (since h(f) is a causal function) 
0 


0 
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That is, 
x(t) =u * h(t) 


Since convolution is commutative, we may also write 


t 


x(t) =h * u(t) = | A(t)u(t — tT) dt 


0 


In summary, we have the result that if the impulse response of a linear time-invariant 
system is A(t) then its response to an arbitrary input w(f) is 
t 


(= | u(t)h(t — tT) dt= | A(t)u(t — tT) dt (5.83) 


0 0 


It is important to realize that this is the response of the system to the input u(t) assuming 
it to be initially in a quiescent state. 


Example 5.60 The response 6,(f) of a system to a driving force 6,(f) is given by the linear differential 


equation 
2 
oo, 2804 56, = 6, 
dre dt 


Determine the impulse response of the system. Hence, using the convolution integral, 
determine the response of the system to a unit step input at time ¢ = 0, assuming that it 
is initially in a quiescent state. Confirm this latter result by direct calculation. 


Solution The impulse response A(f) is the solution of 
dh. dh 


7 a 


subject to the initial conditions 4(0) = h(0) = 0. Taking Laplace transforms gives 
(s° + 2s + 5)H(s) = L{6(H} = 1 
so that 
H(s) = = =} 4 
s +25+5 (s+1) +2 
which, on inversion, gives the impulse response as 
h(t) = $e‘ sin 2t 


Using the convolution integral 


0 


6,(t) = | A(t)O(t — T) dt 
with 0,(t) = LH(f) gives the response to the unit step as 


0,(t) = :| e’sin2tdt 


0 
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Integrating by parts twice gives 


t 


6,(t) = —} e* sin 2t — e“* cos 2t + 1 - 2| e' sin2tdt 


0 
= —}e‘sin 2t — e‘cos2t + 1-40, 
Hence 
0,(t) = 5 (1 — cos 2t — te‘ sin 27) 


(Note that in this case, because of the simple form of 0,(f), the convolution integral 
fi, h(t) 0.(t — T) dt is taken in preference to f0,(t)A(t — T) dt.) 

To obtain the step response directly, we need to solve for t = 0 the differential 
equation 


subject to the initial conditions 0,(0) = 6,(0) = 0. Taking Laplace transforms gives 
(2 +25+5)Q(s)= 4 
s 


so that 


9 =—__1—_ - 1 8+2 
s(s +25+5) s ~(s+1) +4 


which, on inversion, gives 
6,(t) = } — $e“(cos 2t+ $ sin 24) = 1 (1 — e*cos 2t— } e“ sin 22) 


confirming the previous result. 


We therefore see that a linear time-invariant system may be characterized in the 
frequency domain (or s domain) by its transfer function G(s) or in the time domain by 
its impulse response A(t), as depicted in Figures 5.50(a) and (b) respectively. The 
response in the frequency domain is obtained by algebraic multiplication, while the 
time-domain response involves a convolution. This equivalence of the operation of 
convolution in the time domain with algebraic multiplication in the frequency domain 
is clearly a powerful argument for the use of frequency-domain techniques in 
engineering design. 


Us) X(s) u(t) x(t) 
X(s) = G(s) U(s) x(t) = u* h(t) 
(a) (b) 


Figure 5.50 (a) Frequency-domain and (b) time-domain representations of a linear 
time-invariant system. 
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5.6.8 Exercises 


For the following pairs of causal functions f(t) and 
g(t) show that f« g(f) = g* f(0): 


(a) fM=t, g(t) = cos 3t 
(b) f=t+1, g()=e* 
(c) fO=0, = g(f) = sin2r 
(d) f=e"%, g(t) =sint 


Using the convolution theorem, determine the 
following inverse Laplace transforms. Check your 
results by first expressing the given transform in 
partial-fractions form and then inverting using the 
standard results: 


(a) gi | 
s(s+3) 


(b) $6 | ed | 
(s-2)°(s+3) 


ors 
s(s+4) 


Taking f(A) = A and g(A) =e, use the inverse form 
(5.80) of the convolution theorem to show that the 
solution of the integral equation 


y(t) = | Ae dr 


0 


51 


By2 


is 
y(t) = (t- 1) + e%. 


Find the impulse response of the system 
characterized by the differential equation 


2 my 
dx 78k 4 12x = W(0) 
dt dt 


and hence find the response of the system to the 
pulse input u(t) = ALH(t) — A(t — T)], assuming that 
it is initially in a quiescent state. 


The response 6,(t) of a servomechanism to a driving 
force 0,(t) is given by the second-order differential 
equation 
2 
60 5 4o 4 59, = 6, (t=0) 
dr dt 
Determine the impulse response of the system, 
and hence, using the convolution integral, obtain 
the response of the servomechanism to a unit step 
driving force, applied at time ¢ = 0, given that the 
system is initially in a quiescent state. 
Check your answer by directly solving the 
differential equation 
2 
18, + 4990 + 50,= 1 
dt dt 
subject to the initial conditions 6,= 6, = 0 
when ¢=0. 


Solution of state-space equations 


In this section we return to consider further the state-space model of dynamical systems 
introduced in Section 1.9. In particular we consider how Laplace transform methods 
may be used to solve the state-space equations. 


5.7.1 SISO systems 


In Section 1.9.1 we saw that the single input—single output system characterized by the 
differential equation (1.66) may be expressed in the state-space form 


x = Ax + bu 


yoex 


(5.84a) 
(5.84b) 
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Solution 
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where x = x(f) = [x, x, .. . x,]' is the state vector and y the scalar output, the correspond- 
ing input-output transfer function model being 


YG) =. Pas tty Se 


G(s) = = 
U(s)  s"+a,.8" '+...¢a, <A 


(5.85) 


where Y(s) and U(s) are the Laplace transforms of y(f) and u(t) respectively. Defining 
A and bas in (1.60), that is, we take A to be the companion matrix of the left-hand side 
of (1.66) and take b=[00...0 1]". In order to achieve the desired response, the vector 
c is chosen to be 


= [bp b)4-+b05..0) (5.86) 

a structure we can confirm to be appropriate using Laplace transform notation. Defining 
Xs) = L£{x,(t)} and taking 
1 


n-1 


X(s) = 5 
S +4, 18" +-+++a 


U(s) 


we have 
X,(s) = sX,(s), X,(s) = 8X,(s) = s°X,(s), ... , X,(8) = sX,1(s) = s"'X,(s) 
so that 
Y(s) = boX\(s) + b,X{(s) +... +b, Xnail) 
Braet bas +o bu" U5) 
S FQ “bse ao 


which confirms (5.86). 

Note that adopting this structure for the state-space representation the last row in A 
and the vector c may be obtained directly from the transfer function (5.85) by reading 
the coefficients of the denominator and numerator backwards as indicated by the 
arrows, and negating those in the denominator. 


For the system characterized by the differential equation model 
dy, dy dy du , du 
+664 l1~+3y = 54+ + 5.87 
dp dxdt Cg? at ee 
considered in Example 1.41, obtain 


(a) atransfer function model; 
(b) a state-space model 


(a) Assuming all initial conditions to be zero, taking Laplace transforms throughout 
in (5.87) leads to 


(s° + 65° + 11s + 3)Y¥(s) = (5s* +5 + 1)U(s) 


so that the transfer-function model is given by 


6) = Y(s) = Se eel ec 
Us) s'+6s°+11ls+3 <A 
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0 1 O 0 
(b) Taking A to be the companion matrix A =| 0 0 1| and b = J0) then 
-3 -ll -6 1 


c=[1 1 5]' and the corresponding state-space model is given by (5.84). 


Note: The eigenvalues of the state matrix A are given by the roots of the charac- 
teristic equation | Al— A] = 2 + 6# + 114+3 =0, which are the same as the poles 
of the transfer function G(s). 


Defining 


L{x,(t)} X,(s) 


Lix(t)}) _ |X2(s) 


L{x(1)} = = X(s) 


L{x,(t)} X,(8) 
and then taking the Laplace transform throughout in the state equation (5.84a) gives 
sX(s) — x(0) = AX(s) + bU(s) 
which on rearranging gives 
(sl — A)X(s) = x(0) + bU(s) 
where I is the identity matrix. Premultiplying throughout by (sl — A)! gives 
X(s) = (sl— A) 'x(0) + (sl — A)'bU(s) (5.88) 


which on taking inverse Laplace transforms gives the response as 
x(t) = £1 (sl— AY! }x(0) + Lf (sl — AY bU(s)} (5.89) 


Having obtained an expression for the system state x(f) its output, or response, y(f) may 
be obtained from the linear output equation (5.84b). 
Taking the Laplace transform throughout in (5.84b) gives 


Y¥(s) = c'X(s) (5.90) 
Assuming zero initial conditions in (5.88) we have 

X(s) = (sl— A)'bU(s) 
which, on substitution in (5.90), gives the input-output relationship 

¥(s) = c'(sl— A) 'bU(s) (5.91) 


From (5.91) it follows that the system transfer function G(s) may be expressed in the form 


~ ols Ay'p = CadiGt = A)b 
G(s) = c'(sl— A)'b det(sl A) 


which indicates that the eigenvalues of A are the same as the poles of G(s), as noted at 
the end of Example (5.61). It follows, from Definition 5.2, that the system is stable 
provided all the eigenvalues of the state matrix A have negative real parts. 
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Solution 
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On comparing the solution (5.89) with that given in (1.81), we find that the transition 


matrix @(t) = e*’ may also be written in the form 
Ot) = L1{(st— AY} 

As mentioned in Section 1.10.3, having obtained ®(f), 
P(t, to) =e 


A(t-t9) 


may be obtained by simply replacing t by t — f. 


Using the Laplace transform approach, obtain an expression for the state x(f) of the 


system characterized by the state equation 


wit eel 
%,(f) 1-3} | x2(t) 1 
when the input u(t) is the unit step function 


0 (t<0) 


u(t) = H(t) -{\ fg 350) 


and subject to the initial condition x(0)=[1  1]’. 


In this case 


ae[ I} a ut)=H(t), X= 1 
i =a 1 


aae[**! at det(sl — A) = (s + 1)(s +3) 


-l s+3 
giving 
wie 
(sI- A)! = 1 s+3 0 = s+] 
(st+1)(s+3)) 1 s41 1 1 


0 


1 


2s+1) 2(s-3) st+3 


which, on taking inverse transforms, gives the transition matrix as 


=f 
e 0 
-3t —3t 
e e 


e“ = £'{(sl-A)'}= 


so that the first term in the solution (5.89) becomes 


£"f(sl-A)"}x) = a a 
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Since U(s) = L{H(t)} = I/s, 


po pueye— sles © |fal 
(sl - A) 1) = ET 1 llth 


= 1 st+3 
~ -s(s+ 1)(s+3)| 542 


[ee 
_ s stl 
2 1 7 1 
3s 2(s+1) 6(s+3) 


so that the second term in (5.89) becomes 


LF '{(sl-A)'bU(s)} = (5.93) 


5.7.2 Exercises 


53 A system is modelled by the following differential 
equations 


X, + 5x, +x) = 2u 
X_ — 3x, +X) = Su 56 


coupled with the output equation 
yHx, + 2x, 
Express the model in state-space form and obtain 


the transfer function of the system. 


54 Find the state-space representation of the second 
order system modelled by the transfer function 


G(s) = Y(s) — : s+l 
U(s) 5° +75 +6 


55 Obtain the dynamic equations in state-space form 
for the systems having transfer-function models 


sv +3542 
s+4s°+3s 


8 +3545 
S+6s +5547 


(a) (b) 


using the companion form of the system matrix in 
each case. 


In formulating the state-space model (5.84) it is 
sometimes desirable to specify the output y to 

be the state variable x,; that is, we take 

c=[l 0 O]'. If A is again taken to be 
the companion matrix of the denominator then it 
can be shown that the coefficients b,, b,,..., 6b, of 
the vector b are determined as the first 7 coefficients 
in the series in s' obtained by dividing the 
denominator of the transfer function (5.85) into the 
numerator. Illustrate this approach for the transfer- 
function model of Figure 5.51. 


U(s) 5s27+541 Ms) 
s3 + 652 + lls +6 


Figure 5.51 Transfer-function model of Exercise 56. 
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A system is governed by the vector—matrix 
differential equation 


K(f) = : 4 ct) + ne (t) (¢29) 
x = ) 1 1 ir = 


where x(f) and u(f) are respectively the state 
and input vectors of the system. Use Laplace 
transforms to obtain the state vector x(t) for the 
input u(t)=[4 3]' and subject to the initial 
condition x(0)=[1 2]. 


Given that the differential equations modelling a 
certain control system are 

X, =x, -—3x,+u 

X= 2x, - 4x, +u 


use (5.89) to determine the state vector 

x=[x, x,]" for the control input u = e~, 
applied at time ¢ = 0, given that x, = x, = 1 at time 
t=0. 


Using the Laplace transform approach, obtain 
an expression for the state x(¢) of the system 
characterized by the state equation 


lk 0 Iilx| f2 
x= = +] 0 fu 
Xo —2 -3|| xX, 0 
where the input is 


es | (t < 0) 
e’ (t=0) 


and subject to the initial condition x(0)=[1 OJ". 


5.7.3 MIMO systems 
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60 


61 


A third-order single-input—single-output system is 
characterized by the transfer-function model 


Y(s) ___3s87+2s+1 
Us) +65’ +11ls+6 


Express the system model in the state-space form 


x=Ax+ bu (5.94a) 


y=ex (5.94b) 


where A is in the companion form. By making a 
suitable transformation x = Mz, reduce the state- 
space model to its canonical form, and comment 
on the stability, controllability and observability 
of the system. 

Given that 


(i) anecessary and sufficient condition for 
the system (5.94) to be controllable is 
that the rank of the Kalman matrix 
[b Ab A’b A’'b] be the same 
as the order of A, and 

(ii) a necessary and sufficient condition for it to 
be observable is that the rank of the Kalman 
matrix [c A'’c (A‘)’c (A’)"'c] be 
the same as the order of A, 


evaluate the ranks of the relevant Kalman matrices 
to confirm your earlier conclusions on the 
controllability and observability of the given 
system. 


Repeat Exercise 60 for the system characterized by 
the transfer-function model 


yr 43545 
s +6s +55 


As indicated in (1.69) the general form of the state-space model representation of an 
nth-order multi-input—multi-output system subject to r inputs and / outputs is 


x=Ax+ Bu 
y=Cx+ Du 


(5.95a) 
(5.95b) 


where x is the n-state vector, u is the r-input vector, y is the /-output vector, A is the 
n Xn system matrix, B is the n x r control (or input) matrix and C and D are respectively 
1x nand/x r output matrices, with the matrix D relating to the part of the input that is 


applied directly into the output. 
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Defining 
Fy(y}  [Yals) 
yore a are) 
Lty()}] — |¥(s) 
Luly] [Us 
squcayy = [OW 5 (AO. 1) 


L{u,(t)} U,(s) 
and taking Laplace transforms throughout in the state equation (5.95a), following the 
same procedure as for the SISO case, gives 
X(s) = (sl— A) 'x(0) + (sl -— A)'BU(s) (5.96) 
Taking inverse Laplace transforms in (5.96) gives 
x(t) = £'{(sl— A) '}x(0) + L£'{(sl- A) 'BUs)} (5.97) 


The output, or response, vector y(¢) may then be obtained directly from (5.95b). 

We can also use the Laplace transform formulation to obtain the transfer matrix 
G(s), between the input and output vectors, for a multivariable system. Taking Laplace 
transforms throughout in the output equation (5.95b) gives 


Y(s) = CX(s) + DU(s) (5.98) 
Assuming zero initial conditions in (5.96) we have 

X(s) = (sl— A)'BUs) 
Substituting in (5.98), gives the system input-output relationship 

Y(s) = [C(sl— A)'B + D]JU(s) 


Thus the transfer matrix G(s) model of a state-space model defined by the quadruple 
{A, B, C, D} is 


G(s) = C(sI- AY'B+D (5.99) 
The reverse problem of obtaining a state-space model from a given transfer matrix 
is not uniquely solvable. For example, in Section 1.10.6 we showed that a state-space 
model can be reduced to canonical form and indicated that this was without affecting the 


input—output behaviour. In Section 1.10.6 it was shown that under the transformation 
x = Tz, where T is a non-singular matrix, (5.95) may be reduced to the form 


2=Az+Bu 
pte De (5.100) 
where z is now a state vector and 


A=T"'AT, B=T"'B, C=CT,D=D 


www.z2Ofile.org 


Example 5.63 


Figure 5.52 Network 
of Example 5.63. 


Solution 
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From (5.99), the input-output transfer matrix corresponding to (5.100) is 

G(s) = E(sl- A)'B+ D 
=CT(sl- T'AT)'T'B+D 
=CT(sT'T-T'AT)'T'B+D 
=CT[T (sI-A)T]'T'B+D 
=CT[T '(sl- A)'T]T'B+D_ (using the commutative property) 
=C(slI-A)'B+D 
= G(s) 


where G(s) is the transfer matrix corresponding to (5.95), confirming that the input-output 
behaviour of the state-space model defined by the quadruple {A, B, C, D} is the same as 
that defined by the quadruple {A, B, C, D}. The problem of finding state-space models 
that have a specified transfer-function matrix is known as the realization problem. 

It follows from (5.99) that 


_ Cadj(sl- A)B 
Se = ia 


Clearly, if s = p is a pole of G(s) then it must necessarily be an eigenvalue of the state 
matrix A, but the converse is not necessarily true. It can be shown that the poles of G(s) 
are identical to the eigenvalues of A when it is impossible to find a state-space model 
with a smaller state dimension than having the same transfer-function matrix. In such 
cases the state space model is said to be in minimal form. 


(a) Obtain the state-space model characterizing the network of Figure 5.52. Take the 
inductor current and the voltage drop across the capacitor as the state variables, 
take the input variable to be the output of the voltage source, and take the output 
variables to be the currents through L and R, respectively. 


(b) Find the transfer-function matrix relating the output variables y, and y, to the 
input variable w. Thus find the system response to the unit step u(¢) = H(t), assuming 
that the circuit is initially in a quiescent state. 


(i) u=ein Ry =5.5Q 
1= 5.5 


(a) The current i, in the capacitor is given by 
ig = (0g = CX, 
Applying Kirchhoff’s second law to the outer loop gives 


€=R(i, tic) +c t+ Roig = Ri + CX) +x, + ROCK, 
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leading to 


oe eee. ae ee oe 
C(R, +R) | C(R, +R) C(R, + Rp) 


X= 


Applying Kirchhoff’s second law to the left-hand loop gives 


e=R (i, +i.) + Li, = R(x, + Ck,) + LX, 


leading to 

nek RR pe 
L(R, +R) L(R, +R) LR,+R, 

Also, 

Vi =X 

y2 = CX, = : as +—* 


= x) 7 Xo 
R, +R, R,+R, R, +R, 


Substituting the given parameter values leads to the state-space representation 


yi l 0 1 x, 0 
ae, =e a aee 
}y2] 0 Ls 15 | | *2 15 


which is of the standard form 


X= Ax + bu 
y=Cx+ du 


(b) From (5.99), the transfer-function matrix G(s) relating the output variables y, and 
y> to the input u is 


G(s) = C(sl—- A)'b+d 


Now 
sl-A= s+2 4 
—2 still 
giving 
(I-A) = 1 [stl -4 
(st+3)(s+10)| 2 st+2 


7 1 fo) 03630 1 Jfst11 9-4 Jf2 
col-A) 56> ——_—_ 
(s+3)(s+10)}-2 —-4 2 s42||2 


1 Us+15 


~ (s+ 3)(s+10)| 26,4 
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so that 
; Us+15 
G(s) = 1] 87 PO] =] +3) +10) 
(s+3)(s +10) —%5-4 2 =He=4 ; 


ht 
(s+3)(st10). 


The output variables y, and y, are then given by the inverse Laplace transform of 
¥(s) = G(s)U(s) 
where U(s) = £[u(t)] = LLH(O)] = 1/s; that is, 


Ost 15 
s(s+3)(s+ 10) 
—25-4 
_ 4, 
s(s+3)(s+10) 15s 


Y(s)= 


ue 4 

14 = 7 
s+3 s+10 
2 2 4 2: 
§_ 34, _21_ 48 
s gst+3 s+10 5s 


i 
2 
S 


which on taking inverse Laplace transforms gives the output variables as 


1, 1,73 _ 4 ,710t 
Vi 2+ ge 7 
a (t = 0) 
2 .-3t 4 10t 
V2 —3@ Fai ® 


In MATLAB the function t £2ss can be used to convert a transfer function to state- 
space form for SISO systems. At present there appears to be no equivalent function 
for MIMO systems. Thus the command 


lA, B,C, (Dll = EE2Ss (lo, a) 
returns the A, B, C, D matrices of the state-form representation of the transfer function 


dl 
bis +--+. +b,.,8+b, 


G(s) = C(6i— A) B+ D = 


GS 4 oc e Elen eh 


where the input vector a contains the denominator coefficients and the vector b 
contains the numerator coefficients, both in ascending powers of s. 

(Note: The function t fss can also be used in the case of single-input—multi-output 
systems. In such cases the matrix numerator must contain the numerator coefficients 
with as many rows as there are outputs.) 

To illustrate consider the system of Example 5.61, for which 


5s +541 


CS) ee ne 
s +6s°+11s+3 
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In this case the commands: 


b al et EI 
a & [i © Ih Bile 
[A, B,C, D] = tu2ss (lo, a) 


return 

we SG sili; = 
1 0 @ 
0 i @ 

Bye aeell 
0 
0 

(GO) ee 5) Ale lk 

D = 0 


giving the state-space model 


x; -6 -11 -3)|% 1 Xx 
2) (i (00) ce) ca 0) cea ai 
is 0 1 O}}x; 0 35 


(Note: This state-space model differs from the one given in the answer to Example 
5.61. Both forms are equivalent to the given transfer function model, with an alter- 
native companion form taken as indicated in Section 1.9.1.) 

Likewise, in MATLAB the function ss2t £ converts the state-space representa- 
tion to the equivalent transfer function/matrix representation (this being applicable 
to both SISO and MIMO systems). The command 


Ud,2| = SSID (A, B,C, Dy, aw) 
returning the transfer function/matrix 
G(s) = C(sl- A)'B+D 


from the iu-th input. Again the vector a contains the coefficients of the denominator 
in ascending powers of s and the numerator coefficients are returned in array b with 
as many rows as there are outputs. 

(Note: The entry iu in the command can be omitted when considering SISO 
systems so, for example, the commands: 


[=O =li =37 10 Of @ 2 Ole 


, 


A 
1h 
E 
D 
19), QI HSSSZtc i (VA, 183, (C , 1D) 


Ip = 0 5, CO0OO 1.0000 1.0000 
a = L_OOOO Ss OWOO) iil WONOO) 3) WOOK) 
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giving the transfer function representation 


2 
Goa) = 5s st 
s +6s°+11s+3 


which confirms the answer to the above example. As an exercise confirm that the 
state-space model obtained in the answer to Example 5.61 is also equivalent to this 


transfer function representation.) 


To illustrate a MIMO system consider the system in Exercise 63, in which the 


state-space model is 


x G1 0) olla) for 0 
Ho] _|-1 -1 0 Iles) 1 ‘lh 
x3 0 @® @ 1] | x3 0 Ol) 
Men 0 I =f tll pe, (Ol 

x) 
y]  f1 0 0 olla 
y| |0 0 1 { sy 


and we wish to determine the equivalent transfer matrix. The commands: 


A = 10 a © Oso =i © Is0 © @ ie@ © =i sills 
BelO Opi Oro OpO iL) ¢ 

Cal OO GO pO O 1 Wl, 

D=(0 @ 70 Olly 

ol,all = SSeAte (A, 18, (Cw), 1) 


return the response to 2, 


Io = © @ LeWOWl) LOOM) 1 Woo 
0 0.0000 0.0000 1.0000 0.0000 
6 = 1.0000 2.0000 2.0000 2.0000 1.000 


and the additional command 
[s2 ,2]| = Sec VA, 18), (Cy Dy, A) 
returns response to w/, 


lz = © ©0000 ©.0000 1.0000 ©0000 
0 0.0000 1.0000 1.0000 1.0000 
A = 1_OWOO 2.0000 2.0000 2-CO0M 1.0000) 


leading to the transfer matrix model 


2 
1 Ss +ts41 S 
GS ere ee 
Sse DS sk AS se Bese Il s SoG Il 


2 1 s+s+1 s 
(s+1)(s°+1)| 5 e+e4l 
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62 


63 


64 


5.7.4 Exercises 


Determine the response y = x, of the system 
governed by the differential equations 


X, =—2x,+u,- uy (t = 0) 


Xy =X, - 3x). +u, 4+ Uy 


to an input w=[u, w]'=[1  f]' and subject to 
the initial conditions x,(0) = 0, x,(0) = 1. 


Consider the 2-input—2-output system modelled by 
the pair of simultaneous differential equations 


VAY -WtV = My 


V2 + ¥2- Yi + Yo = Uy 


Taking the state vector tobex=[y, }; 2 Jr’ 
express the model as a state-space model of the form 


x=Ax+ Bu 
y= 


Determine the transfer matrix and verify that its 
poles are identical to the eigenvalues of the state 
matrix A. 


Considering the network of Figure 5.53 
(a) Determine the state-space model in the form 
x=Ax+ Bu 


y= 


uy, =j 


Figure 5.53 Network of Exercise 64. 


Take the inductor currents in L,, L, and L; as 
the state variables x,, x2, x; respectively; take 
the input variables u, and uw, to be the outputs 
of the current and voltage sources respectively; 
and take the output variables y, and y, to be the 
voltage across R, and the current through L, 
respectively. 


(b) Determine the transfer matrix G(s) relating the 


output vector to the input vector. 


(c) Assuming that the circuit is initially in a 
quiescent state, determine the response y(f) 
to the input pair 


u(t) = A) 
u(t) = tH(t) 


where H(t) denotes the Heaviside function. 


5.8 Engineering application: Bic (titans teins 


Frequency-response methods provide a graphical approach for the analysis and design 
of systems. Traditionally these methods have evolved from practical considerations, 
and as such are still widely used by engineers, providing tremendous insight into over- 
all system behaviour. In this section we shall illustrate how the frequency response can 
be readily obtained from the system transfer function G(s) by simply replacing s by ja. 
Methods of representing it graphically will also be considered. 
Consider the system depicted in Figure 5.41, with transfer function 


G(s) = 


(s- pi)(S- po)... (8 Pn) 


When the input is the sinusoidally varying signal 


u(t) = A sin ot 
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applied at time ¢ = 0, the system response x(t) for t = 0 is determined by 
X(s) = G(s) L{A sin wt} 
That is, 


X(s) = Gs) 42, 
Ss +@ 


= KAQ@(s - 2,)(8 — 22)... (8 = Zn 
(s-pi)(S~ Pr) . - - (8 Pa)(S - j@)(s + jo) 
which, on expanding in partial fractions, gives 


X(s) = —— + — 
ee =a ae Dees Di 


where @,, %, B,, Bo, ..., B, are constants. Here the first two terms in the summation are 

generated by the input and determine the steady-state response, while the remaining 

terms are generated by the transfer function and determine the system transient response. 
Taking inverse Laplace transforms, the system response x(f), tf = 0, is given by 


n 
xH=ae%+a,e%+ PY Be” (t= 0) 
i=] 


In practice we are generally concerned with systems that are stable, for which the poles 
Pp, i= 1,2,..., n, of the transfer function G(s) lie in the left half of the s plane. 
Consequently, for practical systems the time-domain terms f,e”", i= 1, 2,...,n, decay 
to zero as ¢ increases, and will not contribute to the steady-state response x,,(t) of the 
system. Thus for stable linear systems the latter is determined by the first two terms as 


x(t) = ae!" + a,” 


Using the ‘cover-up’ rule for determining the coefficients a, and @, in the partial- 
fraction expansions gives 


ee ane - 46,0) 


(= jay(s+jo)| 


(s= jo)(s+ jo) | 


Oy = le ers) 7 “$6 io) 
J 


so that the steady-state response becomes 
x,(#) = 4 G(ja) ei — 4 Gja) (5.102) 
2j 2j 


G(j@) can be expressed in the polar form 
G(ja) = |GCjo)| e864 


where |G(jq@)| denotes the magnitude (or modulus) of G(j@). (Note that both the 
magnitude and argument vary with frequency @.) Then, assuming that the system has 
real parameters, 


G(-j0) = |G ja)] e249 
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Solution 


and the steady-state response (5.102) becomes 
= A ‘ jarg G(j@)] pjot A : —j arg G(j@)] ,—jot 
X3s(t) = = [|G(ja)| ee!) e’ — = [|G(ja)le Je 
2j 2j 
A 5 jl@t+arg G( ja) —j[otarg G( ja) 
= —|G(jo)|[e-! gG(j 1) _ ell gG(j bi 
2j 


That is, 
x,(t) = A |G(jo)| sin [wt + arg G(jo)] (5.103) 


This indicates that if a stable linear system with transfer function G(s) is subjected to a 
sinusoidal input then 


(a) the steady-state system response is also a sinusoid having the same frequency @ 
as the input; 

(b) the amplitude of this response is |G(jq@)| times the amplitude A of the input 
sinusoid; the input is said to be amplified if |G(j@)| > 1 and attenuated if 
|G(jo)| < 1; 

(c) the phase shift between input and output is arg G(j@). The system is said to lead 
if arg G(j@) > 0 and lag if arg G(j@) < 0. 


The variations in both the magnitude |G(jq@)| and argument arg G(jq@) as the fre- 
quency @ of the input sinusoid is varied constitute the frequency response of the 
system, the magnitude | G(j@)| representing the amplitude gain or amplitude ratio of 
the system for sinusoidal input with frequency @, and the argument arg G(j@) represent- 
ing the phase shift. 

The result (5.103) implies that the function G(j@) may be found experimentally by 
subjecting a system to sinusoidal excitations and measuring the amplitude gain and 
phase shift between output and input as the input frequency is varied over the range 
0 < w@< .~., In principle, therefore, frequency-response measurements may be used to 
determine the system transfer function G(s). 

In Chapters 7 and 8, dealing with Fourier series and Fourier transforms, we shall see 
that most functions can be written as sums of sinusoids, and consequently the response 
of a linear system to almost any input can be deduced in the form of the corresponding 
sinusoidal responses. It is important, however, to appreciate that the term ‘response’ in 
the expression ‘frequency response’ only relates to the steady-state response behaviour 
of the system. 

The information contained in the system frequency response may be conveniently 
displayed in graphical form. In practice it is usual to represent it by two graphs: one 
showing how the amplitude |G(jq@)| varies with frequency and one showing how the 
phase shift arg G(j@) varies with frequency. 


Determine the frequency response of the RC filter shown in Figure 5.54. Sketch the 
amplitude and phase-shift plots. 


The input—output relationship is given by 


1 
E a Oe 
of) RCs +1 i(s) 
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R so that the filter is characterized by the transfer function 


e(0) ae eolt) G0) =a 
S+ 


Figure 5.54 RC filter. | Therefore 


cr _ 1-jRCo 
RCjO+1 14R°Co 


1 . RC@ 


SS 
l¢RCa “14R'C au 


giving the frequency-response characteristics 


amplitude ratio = | G(ja)| 


= 4). 1 4, ew 
(+RCa)y (14+RCoy 


1 
J\1+R°C@) 


phase shift = arg G(j@) = —tan”' (RC@) 
Note that for @ = 0 
IG(Qjo)|=1, arg G(jo) = 0 
and as @ > © 
|G(j@)| > 0, arg G(j@) > -i1 


Plots of the amplitude and phase-shift curves are shown in Figures 5.55(a) and (b) 
respectively. 


Figure 5.55 arg G(jw) 

Frequency-response 90° 

plots for Example 5.64: 

(a) amplitude plot; 

(b) phase-shift plot. fe) 
-45° 
—90° 


(b) 


For the simple transfer function of Example 5.64, plotting the amplitude and phase- 
shift characteristics was relatively easy. For higher-order transfer functions it can be 
a rather tedious task, and it would be far more efficient to use a suitable computer 
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Example 5.65 


Solution 


package. However, to facilitate the use of frequency-response techniques in system 
design, engineers adopt a different approach, making use of Bode plots to display 
the relevant information. This approach is named after H. W. Bode, who developed 
the techniques at the Bell Laboratories in the late 1930s. Again it involves drawing 
separate plots of amplitude and phase shift, but in this case on semi-logarithmic graph 
paper, with frequency plotted on the horizontal logarithmic axis and amplitude, or phase, 
on the vertical linear axis. It is also normal to express the amplitude gain in decibels 
(dB); that is, 


amplitude gain in dB = 20 log|G(j@)| 
and the phase shift arg G(j@) in degrees. Thus the Bode plots consist of 


(a) aplot of amplitude in decibels versus log @, and 

(b) a plot of phase shift in degrees versus log @. 

Note that with the amplitude gain measured in decibels, the input signal will be 
amplified if the gain is greater than zero and attenuated if it is less than zero. 

The advantage of using Bode plots is that the amplitude and phase information can 
be obtained from the constituent parts of the transfer function by graphical addition. It 
is also possible to make simplifying approximations in which curves can be replaced by 
straight-line asymptotes. These can be drawn relatively quickly, and provide sufficient 
information to give an engineer a ‘feel’ for the system behaviour. Desirable system 
characteristics are frequently specified in terms of frequency-response behaviour, and 
since the approximate Bode plots permit quick determination of the effect of changes, 
they provide a good test for the system designer. 


Draw the approximate Bode plots corresponding to the transfer function 


= 4x 10°(5+5) 
OCS) = 100+ 5)(20+5) (5.104) 


First we express the transfer function in what is known as the standard form, namely 


10(1 + 0.2s 
G(s) = L0G + 0.25) 
(3) = 7 40.01s)(1 + 0.058) 


giving 
. 10(1 +40.2@ 
G(i@) -< ——10G. 410.20) 
(10) = (1 + 0.01@)(1 + j0.05a) 
Taking logarithms to base 10, 


20 log|G(j@)| = 20 log 10 + 20 log |1 + j0.2@| — 20 log |ja| 
— 20 log |1 + j0.01@| — 20 log |1 + j0.05a| 
arg G( j@) = arg 10 + arg (1 + j0.2@) — arg jw — arg (1 + j0.01@) 
—arg(1 + j0.05@) (5.105) 


The transfer function involves constituents that are again a simple zero and simple 
poles (including one at the origin). We shall now illustrate how the Bode plots can be 
built up from those of the constituent parts. 
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20 log I + jel 


Corner frequency 


we l/r F f° 10 
| A ¥da| 20 dB/decade : 
a" -5 


2 


Consider first the amplitude gain plot, which is a plot of 20 log | G( j@)| versus log @: 


(a) fora simple gain k a plot of 20 log is a horizontal straight line, being above the 
0 dB axis if k > 1 and below it if k < 1; 


(b) for a simple pole at the origin a plot of —20 log @ is a straight line with slope 
—20 dB/decade and intersecting the 0 dB axis at w= 1; 


(c) fora simple zero or pole not at the origin we see that 


: 0 as @ > 0 
20 log|1+jt@| > 
20 log tw = 20 logwm- 20 log(1/t) as@—> © 


Note that the graph of 20 log Ta is a straight line with slope 20 dB/decade and inter- 
secting the 0 dB axis at @= 1/t. Thus the plot of 20 log|1 + jt@| may be approximated 
by two straight lines: one for @ < 1/tTand one for @ > 1/t. The frequency at intersection 
@= 1/Tis called the breakpoint or corner frequency; here |1 + jt@| = 2, enabling the 
true curve to be indicated at this frequency. Using this approach, straight-line approxima- 
tions to the amplitude plots of a simple zero and a simple pole, neither at zero, are 
shown in Figures 5.56(a) and (b) respectively (actual plots are also shown). 


-20 log II + jal 


-10 
ae -15 20 dBidecade” ~~. 
~20 
34 5 10 20 2 20 
wirad s~! colrad s~! 
(a) (b) 


Figure 5.56 Straight-line approximations to Bode amplitude plots: (a) simple zero; (b) simple pole. 


Using the approximation plots for the constituent parts as indicated in (a)—(c) ear- 
lier, we can build up the approximate amplitude gain plot corresponding to (5.104) by 
graphical addition as indicated in Figure 5.57. The actual amplitude gain plot, produced 
using a software package, is also shown. 

The idea of using asymptotes can also be used to draw the phase-shift Bode plots, 
again taking account of the accumulated effects of the individual components making 
up the transfer function, namely that 


(i) _ the phase shift associated with a constant gain k is zero; 


(ii) the phase shift associated with a simple pole or zero at the origin is +90° or —90° 
respectively; 


(111) fora simple zero or pole not at the origin 


0 as @ > 0 


tan” (@T) > 
90° as@—> © 


tan'(@T) = 45° when @t= 1 
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Figure 5.57 

Amplitude Bode 
plots for the G(s) 
of Example 5.65. 


Figure 5.58 
Approximate Bode 
phase-shift plots: 
(a) simple zero; 
(b) simple pole. 


log |G (jw) | 40 


30 


20 


107! 100 10! 102 103 


wlrad s-! 


=—=--== Approximate plot Actual plot 


With these observations in mind, the following approximations are made. For fre- 
quencies @ less than one-tenth of the corner frequency @= 1/T (that is, for @ < 1/10T) 
the phase shift is assumed to be 0°, and for frequencies greater than ten times the 
corner frequency (that is, for @ > 10/T) the phase shift is assumed to be +90°. For 
frequencies between these limits (that is, 1/10t < m < 10/t) the phase-shift plot is 
taken to be a straight line that passes through 0° at w= 1/107, +45° at w= 1/T, and +90° 
at @= 10/tT. In each case the plus sign is associated with a zero and the minus sign with 
a pole. With these assumptions, straight-line approximations to the phase-shift plots for 
a simple zero and pole, neither located at the origin, are shown in Figures 5.58(a) and 
(b) respectively (the actual plots are represented by the broken curves). 

Using these approximations, a straight-line approximate phase-gain plot correspond- 
ing to (5.105) is shown in Figure 5.59. Again, the actual phase-gain plot, produced using 
a software package, is shown. 


arg [1/(1 + jwt)] 


arg (1 + jet) 
90° 


45° 


Q° 


10 t Tt 


(a) (b) 
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Figure 5.59 
Phase-shift Bode 
plot for the G(s) 
of Example 5.65. 


EJ 
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arg G(jw) 90° 


50° 


arg (1 + j0.01w) 


—50° 


~100° 


-150° 


~180° 
107! 10° 10! 102 103 
wirad s~! 


—=».== Approximate plot 


Actual plot 


In MATLAB the amplitude and phase-gain plots are generated using the commands 
Ssitic (757) 
Gaol 3 (se) / (Se CLOOS)) * (20s) )) p 
bode (G) 


In the graphical approach adopted in this section, separate plots of amplitude gain 
and phase shift versus frequency have been drawn. It is also possible to represent the 
frequency response graphically using only one plot. When this is done using the pair of 
polar coordinates (|G(jq@)|, arg G(j@)) and allowing the frequency @ to vary, the resulting 
Argand diagram is referred to as the polar plot or frequency-response plot. Such a 
graphical representation of the transfer function forms the basis of the Nyquist approach 
to the analysis of feedback systems. In fact, the main use of frequency-response methods 
in practice is in the analysis and design of closed-loop control systems. For the unity 
feedback system of Figure 5.45 the frequency-response plot of the forward-path 
transfer function G(s) is used to infer overall closed-loop system behaviour. The Bode 
plots are perhaps the quickest plots to construct, especially when straight-line approx- 
imations are made, and are useful when attempting to estimate a transfer function 
from a set of physical frequency-response measurements. Other plots used in practice 
are the Nichols diagram and the inverse Nyquist (or polar) plot, the first of these 
being useful for designing feedforward compensators and the second for designing 
feedback compensators. Although there is no simple mathematical relationship, it is 
also worth noting that transient behaviour may also be inferred from the various frequency- 
response plots. For example, the reciprocal of the inverse M circle centred on the —1 
point in the inverse Nyquist plot gives an indication of the peak over-shoot in the transient 
behaviour (see, for example, G. Franklin, D. Powell and A. Naeini-Emami, Feedback 
Control of Dynamic Systems, Reading, MA, Addison-Wesley, 1986). 


Investigation of such design tools may be carried out in MATLAB, incorporating 
Control Toolbox, using the command ritool (G). 
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5.9 Engineering application: Beeler 


5.9.1 


5.9.2 


In Chapter | we examined the behaviour of linear continuous-time systems modelled in 
the form of vector-matrix (or state-space) differential equations. In this chapter we have 
extended this, concentrating on the transform domain representation using the Laplace 
transform. In Chapter 6 we shall extend the approach to discrete-time systems using the 
z-transform. So far we have concentrated on system analysis; that is, the question ‘Given 
the system, how does it behave?’ In this section we turn our attention briefly to consider 
the design or synthesis problem, and while it is not possible to produce an exhaustive 
treatment, it is intended to give the reader an appreciation of the role of mathematics in 
this task. 


Poles and eigenvalues 


By now the reader should be convinced that there is an association between system 
poles as deduced from the system transfer function and the eigenvalues of the system 
matrix in state-space form. Thus, for example, the system modelled by the second-order 
differential equation 


dy. dy 
re ae 


has transfer function 
G(s) = ——— 


The system can also be represented in the state-space form 
x= Ax + bu, yous (5.106) 


where 


x= [x, Rls A= 


0 i : z 
; b=[0 lj], c=[l 0] 


It is easy to check that the poles of the transfer function G(s) are at s=—1 and s = S, 
and that these values are also the eigenvalues of the matrix A. Clearly this is an 
unstable system, with the pole or eigenvalue corresponding to s = located in the 
right half of the complex plane. In Section 5.9.2 we examine a method of moving this 
unstable pole to a new location, thus providing a method of overcoming the stability 
problem. 


The pole placement or eigenvalue location technique 


We now examine the possibility of introducing state feedback into the system. To do 
this, we use as system input 


u= KX + Use 
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where k = [k, &,]' and u,,, is the external input. The state equation in (5.106) then 


ext 


becomes 
. 0 1 0 
x=), , [et [(Ayx1 + kpx2) + Were] 
i 3 l 
That is, 
. 0 1 0 
x= x+ Uext 
kit+3 k-} 1 


Calculating the characteristic equation of the new system matrix, we find that the 
eigenvalues are given by the roots of 


KB -(k,- )A-(k+ 4) =0 


Suppose that we not only wish to stabilize the system, but also wish to improve the 
response time. This could be achieved if both eigenvalues were located at (say) A = —5, 
which would require the characteristic equation to be 


#+10A+25=0 
In order to make this pole relocation, we should choose 


—(h- $) = 10, —(k,+ $) =25 


indicating that we take k,= —¥ and k, = -2 . Figure 5.60 shows the original system and 


the additional state-feedback connections as dotted lines. We see that for this example 
at least, it is possible to locate the system poles or eigenvalues wherever we please in 
the complex plane, by a suitable choice of the vector k. This corresponds to the choice 
of feedback gain, and in practical situations we are of course constrained by the need 
to specify reasonable values for these. Nevertheless, this technique, referred to as pole 
placement, is a powerful method for system control. There are some questions that 
remain. For example, can we apply the technique to all systems? Also, can it be extended 
to systems with more than one input? The following exercises will suggest answers to 
these questions, and help to prepare the reader for the study of specialist texts. 


Figure5.60 Feedback sees @- 
connections for eigen- ‘ 
value location. : 


Hex (1) 
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5.9.3 Exercises 


An unstable system has Laplace transfer function 


1 


M9 DED 


Make an appropriate choice of state variables to 
represent this system in the form 


X=Ax+bu, y=e'x 
where 
0 1 
x= by X»] > A = 1 1 
2 2 
b=[0 1], c=[1 oy" 


This particular form of the state-space model in 
which A takes the companion form and b has a 
single | in the last row is called the control 
canonical form of the system equations, and 
pole placement is particularly straightforward 
in this case. 

Find a state-variable feedback control of the 
form u = k'x that will relocate both system poles 
at s = —4, thus stabilizing the system. 


Find the control canonical form of the state-space 
equations for the system characterized by the 
transfer function 


pea, eevee 
(s+1)(s+4) 


G(s) 


Calculate or (better) simulate the step response 

of the system, and find a control law that relocates 
both poles at s =—5. Calculate or simulate the step 
response of the new system. How do the two 
responses differ? 


The technique for pole placement can be adapted 
to multi-input systems in certain cases. Consider 
the system 


x=Ax+Bu, 


where 


68 


69 


Writing Bu = b,u, + byu,, where b} =[1  1]' and 
b,=[0 1)’, enables us to work with each input 
separately. As a first step, use only the input w, 

to relocate both the system poles at s = —5. 
Secondly, use input wu, only to achieve the same 
result. Note that we can use either or both inputs 
to obtain any pole locations we choose, subject of 
course to physical constraints on the size of the 
feedback gains. 


The bad news is that it is not always possible to 
use the procedure described in Exercise 67. In the 
first place, it assumes that a full knowledge of the 
state vector x(f) is available. This may not always 
be the case; however, in many systems this problem 
can be overcome by the use of an observer. For 
details, a specialist text on control should be 
consulted. 

There are also circumstances in which the 
system itself does not permit the use of the 
technique. Such systems are said to be 
uncontrollable, and the following example, which 
is more fully discussed in J. G. Reed, Linear System 
Fundamentals (McGraw-Hill, Tokyo, 1983), 
demonstrates the problem. Consider the system 


f aefh 


with 


Find the system poles and attempt to relocate both 
of them, at, say, s = —2. It will be seen that no gain 
vector k can be found to achieve this. Calculating 
the system transfer function gives a clue to the 
problem, but Exercise 69 shows how the problem 
could have been seen from the state-space form of 
the system. 


In Exercise 60 it was stated that the system 
x=Ax+ bu 
yee'x 


where A is ann Xn matrix, is controllable provided 
that the Kalman matrix 
A’b 


M=[b Ab A”'5] 
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is of rank n. This condition must be satisfied if 
we are to be able to use the procedure for pole 
placement. Calculate the Kalman controllability 
matrix for the system in Exercise 68 and confirm 
that it has rank less than n = 2. Verify that the 
system of Exercise 65 satisfies the controllability 
condition. 


We have noted that when the system equations 
are expressed in control canonical form, the 
calculations for pole relocation are particularly 
easy. The following technique shows how to 
transform controllable systems into this form. 
Given the system 


X=Ax+bu, y=e'x 


calculate the Kalman controllability matrix M, 
defined in Exercise 69, and its inverse M"!. 
Note that this will only exist for controllable 
systems. Set v’ as the last row of M™ and form 
the transformation matrix 
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v A”! 


A transformation of state is now made by 
introducing the new state vector z(¢) = Tx(t), and the 
resulting system will be in control canonical form. 
To illustrate the technique, carry out the procedure 
for the system defined by 


: 8 -2 1 
x= x+ u 


and show that this leads to the system 


“EH 


Finally, check that the two system matrices have 
the same eigenvalues, and show that this will 
always be the case. 


5.10 Review exercises (1-34) 


Check your answers using MATLAB or MAPLE whenever possible. 


iL 


Solve, using Laplace transforms, the following 
differential equations: 


2 
(a) dx 4 4 4 5x = 8 cose 
dt dt 


; dx 
subject tox = — =O att=0 
d dt 


2 
(b) sae 2% 
t t 


subject to x = 1 and dx bat? — 0 
dt 
(a) Find the inverse Laplace transform of 


1 
(s+ 1)(s +2)(s° +25 +2) 


(b) A voltage source Ve“sint is applied across a 
series LCR circuit with L = 1, R=3 and C= 5 , 
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Show that the current i(f) in the circuit 
satisfies the differential equation 


Dey Ps 
di, ydi 


; z +2i = Ve'‘sint 
dt t 


Find the current i(f) in the circuit at time 
t => 0 if i(f) satisfies the initial conditions 
i(0) = 1 and (di/dt)(0) = 2. 


Use Laplace transform methods to solve the 
simultaneous differential equations 


2 
dx 4 5W ay 
dt dt 
ee 
dt dt 
: dx _ dy 
bject tox =y= — = = =O at t= 0. 
subject tox =y qa Oa 0 
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4 


Solve the differential equation 


—+2x=cost 


dx , 9d 
de dt 


subject to the initial conditions x = x) and 

dx/dt =x, at t= 0. Identify the steady-state and 
transient solutions. Find the amplitude and phase 
shift of the steady-state solution. 


Resistors of 5 and 20 Q are connected to the 
primary and secondary coils of a transformer with 
inductances as shown in Figure 5.61. At time ¢ = 0, 
with no currents flowing, a voltage E = 100 V 

is applied to the primary circuit. Show that 
subsequently the current in the secondary circuit is 


20 tite 
(Ee (H/41H2_ (1 nyt) 
y4l1 


5Q 202 
Es 


M=1H 


Figure 5.61 Circuit of Review exercise 5. 


(a) Find the Laplace transforms of 
(i) cos(@t+ @) (ii) e°’ sin (@t + @) 


(b) Using Laplace transform methods, solve the 
differential equation 
2 5 
HE ey, = cos 2t 
dt dt 


given that x = 2 and dx/dt= 1 when t=0. 
(a) Find the inverse Laplace transform of 
s-4 
s'+4s+13 


(b) Solve using Laplace transforms the differential 
equation 


“ + 2y =2(2 + cost + 2 sin?) 
given that y = —3 when t= 0. 


Using Laplace transforms, solve the simultaneous 
differential equations 


= + 5x + 3y =Ssint—2cost 


10 


dll 


12 


2 + 3y + 5x = 6sint—3cost 


where x = 1 and y= 0 when t= 0. 


The charge qg on a capacitor in an inductive circuit 
is given by the differential equation 


2: 
14 4 39024 +2 x 10% = 200 sin 1001 
dt dt 

and it is also known that both g and dq/dt are zero 
when ¢ = 0. Use the Laplace transform method to 
find g. What is the phase difference between the 
steady-state component of the current dg/dt and 
the applied emf 200 sin 100¢ to the nearest 
half-degree? 


Use Laplace transforms to find the value of x 
given that 


dx 3 
4— +6x+y=2sin2t 
a x+y =2sin 


Se eae dy = 36% 
dt dt 


and that x = 2 and dx/dt = —2 when t= 0. 


(a) Use Laplace transforms to solve the 
differential equation 
2 
dO 4 908 +169 = sin 21 
dt dt 


given that 9= 0 and d@/dt = 0 when t= 0. 


(b) Using Laplace transforms, solve the 
simultaneous differential equations 
di ; : 
eo + 2i, + 6i, = 0 


einer 
P= = Sy = 0) 
Peidp ts 
given that i; = 1, i, =0 when t=0. 


The terminals of a generator producing a voltage 
V are connected through a wire of resistance 
Rand a coil of inductance L (and negligible 
resistance). A capacitor of capacitance C 
is connected in parallel with the resistance 
Ras shown in Figure 5.62. Show that the 
current i flowing through the resistance R is 
given by 

rcrti+1%+Ri =v 

dt dt 
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13 


14 


aaa 
v( ) Cc 


Figure 5.62 Circuit of Review exercise 12. 


Suppose that 


(i) V=0 fort <0 and V=E (constant) for t = 0 
Gil) La QkeC 
(iii) CR = 1/2n 
and show that the equation reduces to 
2. 3 
oe + ot +2n'i = ee 
dt dt 
Hence, assuming that i = 0 and di/dt = 0 when 


t = 0, use Laplace transforms to obtain an 
expression for i in terms of ¢. 


Show that the currents in the coupled circuits of 
Figure 5.63 are determined by the simultaneous 
differential equations 


Figure 5.63 Circuit of Review exercise 13. 


di, ae . 
L—+ RG, -i) + Ri =E 
dt G@, — i) q 
di : ta: 
Lay t Rb — RG, —b)=0 
Find 7, in terms of ¢, L, E and R, given that 7, = 0 and 
di,/dt = E/L at t= 0, and show that i, = > E/R for 
large t. What does i, tend to for large t? 


A system consists of two unit masses lying in a 
straight line on a smooth surface and connected 
together to two fixed points by three springs. When 
a sinusoidal force is applied to the system, the 
displacements x,(f) and x,(7) of the respective 
masses from their equilibrium positions satisfy 

the equations 
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dx, ; 
~ =X, — 2x, + sin 2t 
dt 
dx 
= =—2x, +X, 
dt 


Given that the system is initially at rest in the 
equilibrium position (x, = x, = 0), use the Laplace 
transform method to solve the equations for x,(t) 
and x,(¢). 


(a) Obtain the inverse Laplace transforms of 


s +4 (ii vos} 
sy +2s +10 (s- 1)7(s- 2) 
(b) Use Laplace transforms to solve the 
differential equation 


2 
dy 4 Wy are 
dt 


given that y = 4 and dy/dt= 2 when t= 0. 


(a) Determine the inverse Laplace transform of 


en eure 
s- 145 +53 


(b) The equation of motion of the moving coil 
of a galvanometer when a current i is passed 
through it is of the form 

gO 4540 4,29 = 

dr dt K 
where @ is the angle of deflection from the 
‘no-current’ position and n and K are positive 
constants. Given that 7 is a constant and 
6 = 0 = d0/dt when t=0, obtain an expression 
for the Laplace transform of 0. 

In constructing the galvanometer, it is desirable 
to have it critically damped (that is, n = K). 
Use the Laplace transform method to solve the 
differential equation in this case, and sketch the 
graph of 6 against t for positive values of t. 


(a) Given that @ is a positive constant, use the 


second shift theorem to 


(i) show that the Laplace transform of 
sin t H(t — @) is 
ew COS a +s sin & 
sy +1 
(ii) find the inverse transform of 


as 
se 


gs +2545 
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(b) Solve the differential equation 


2 
dy od, 5y = sint — sint H(t — 7) 
dt dt 

given that y = dy/dt=0 when t=0. 


Show that the Laplace transform of the 
voltage v(t), with period T, defined by 


1 (0<t<3T) 
v(t) = ; ‘i v(t +T) = v(t) 
= (sr 7) 
is 
l ile =sT/2 
Vis) = = 
(s) Ss 1 Avent 


This voltage is applied to a capacitor of 100 UF and 
a resistor of 250 Q in series, with no charge initially 
on the capacitor. Show that the Laplace transform 
I(s) of the current i(t) flowing, for t = 0, is 


—sT/2 
€ 


1 I 


LS) =a any Sea 
250(s +40) 1 4.¢7°7? 


and give an expression, involving Heaviside step 
functions, for i(t) where 0 St S 27. For T= 10°, 
is this a good representation of the steady-state 
response of the circuit? Briefly give a reason for 
your answer. 


The response x(t) of a control system to a forcing 
term u(t) is given by the differential equation 


ee 


+2x =u(t) (t=0) 
dt t 


Determine the impulse response of the system, and 
hence, using the convolution integral, obtain the 
response of the system to a unit step u(t) = 1 H(t) 
applied at ¢ = 0, given that initially the system is in 
a quiescent state. Check your solution by directly 
solving the differential equation 


dx. 22x =1 (20) 
dt dt 


with x = dx/dt=0 at t=0. 


A light horizontal beam, of length 5 m and constant 
flexural rigidity E/, built in at the left-hand end 
x = 0, is simply supported at the point x = 4m and 
carries a distributed load with density function 


12kNm"! 
24kNm' 


(0 <x < 4) 


wes = | 
(4x <5) 


2il 


22 


23 


Write down the fourth-order boundary-value 
problem satisfied by the deflection y(x). Solve this 
problem to determine (x), and write down the 
resulting expressions for y(x) for the cases 0 S x 
= 4 and 4 <x <5. Calculate the end reaction and 
moment by evaluating appropriate derivatives of 
y(x) at x = 0. Check that your results satisfy the 
equation of equilibrium for the beam as a whole. 


(a) Sketch the function defined by 
0 (0<t<1) 
fd) =41 (St <2) 
0 (t>2) 


Express f(¢) in terms of Heaviside step 
functions, and use the Laplace transform to 
solve the differential equation 


dx 

Hite = f(t 

ar f(t) 
given that x = 0 at t=0. 

(b) The Laplace transform /(s) of the current i(f) 
in a certain circuit is given by 

Saree eee 

s[Ls + R/(1+ Cs)] 


where £, L, R and C are positive constants. 
Determine (i) lim i(¢) and (ii) lim i(¢). 
to0 too 


I(s) = 


Show that the Laplace transform of the half- 
rectified sine-wave function 


ree he t 
0 


of period 27, is 

eet See 

(1+s°)(1-e™) 
Such a voltage v(t) is applied to a 1 Q resistor and 
a 1H inductor connected in series. Show that the 
resulting current, initially zero, is })-5 f(t— nt), 
where /(f) = (sint — cost + e~')H(t). Sketch a 
graph of the function /(?). 


(a) Find the inverse Laplace transform of 
1/s*(s + 1) by writing the expression in 
the form (1/s*)[1/(s + 1)°] and using the 
convolution theorem. 

(b) Use the convolution theorem to solve the 
integral equation 


y@O=t+ | y(u) cos(t — u) du 


0 
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and the integro-differential equation 


t 
| y"(u)y (t — u) du = y(t) 
0 

where y(0) = 0 and y’(0) = y,. Comment on the 
solution of the second equation. 


A beam of negligible weight and length 3/ carries a 
point load W at a distance / from the left-hand end. 
Both ends are clamped horizontally at the same 
level. Determine the equation governing the 
deflection of the beam. If, in addition, the beam 

is now subjected to a load per unit length, w, 

over the shorter part of the beam, what will then 
be the differential equation determining the 
deflection? 


(a) Using Laplace transforms, solve the 
differential equation 


d'x_ 39 3,277 
aoe wer = (t-a) (a>0) 
dt dt 


where H(f) is the Heaviside unit step function, 
given that x = 0 and dx/dt=0 at t=0. 


(b) The output x(t) from a stable linear control 
system with input sin wf and transfer function 
G(s) is determined by the relationship 


X(s) = G(s) L{sin wt} 


where X(s) = L{x(t)}. Show that, after a long 
time ¢, the output approaches x,(4), where 


nh = Re(2-Gua)) 


Consider the feedback system of Figure 5.64, where 
K is a constant feedback gain. 


G(s) 


Figure 5.64 Feedback system of Review 
exercise 26. 


(a) Inthe absence of feedback (that is, K = 0) is the 
system stable? 

(b) Write down the transfer function G,(s) for the 
overall feedback system. 
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(c) Plot the locus of poles of G,(s) in the s plane 
for both positive and negative values of K. 

(d) From the plots in (c), specify for what 
range of values of K the feedback system is 
stable. 

(e) Confirm your answer to (d) using the 
Routh—Hurwitz criterion. 


(a) For the feedback control system of 
Figure 5.65(a) it is known that the impulse 
response is h(t) = 2e~'sint. Use this to 
determine the value of the parameter a. 

(b) Consider the control system of Figure 5.65(b), 
having both proportional and rate feedback. 
Determine the critical value of the gain K for 
stability of the closed-loop system. 


G(s) 


5) X(s) 
(a) 


(b) 


Figure 5.65 Feedback control systems of 
Review exercise 27. 


A continuous-time system is specified in 
state-space form as 


X(t) = Ax(t) + bu(t) 
y(t) = c"x(t) 


where 


“4 th of 


(a) Draw a block diagram to represent the 
system. 

(b) Using Laplace transforms, show that 
the state transition matrix is given by 


i z Aaa! Goatees 
2 eee ne 2 ens 


= By ot 
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(c) Calculate the impulse response of the system, 
and determine the response y(t) of the system to 
an input u(t) = 1 (t = 0), subject to the initial 
state x(0)=[1 OJ’. 


A single-input—single-output system is represented 
in state-space form, using the usual notation, as 


X(t) = Ax(t) + bu(t) 
y(t) = e'x(t) 


For 


show that 


Ae e ‘(cos f- sin?) 
2e‘sint 


—e‘sint 
e ‘(cost +sin ft) 
and find x(7) given the x(0) = 0 and u(t) = 1 (t= 0). 


Show that the Laplace transfer function of the 
system is 


Y(s me 

H(s) = 2 = e(s1-Ay'b 

(= Teg =elel- A) 

and find H(s) for this system. What is the system 
impulse response? 


A controllable linear plant that can be 
influenced by one input u(t) is modelled by 
the differential equation 


X(t) = Ax(t) + bu(d) 


where x(f) = [x,(4)_ x(t) 2 @)|nas 
the state vector, A is a constant matrix with 
distinct real eigenvalues A,,2,,..., 2,, and 
N=, b,]' is a constant vector. 
By the application of the feedback control 


u(t) = Kvzx(0) 


where v, is the eigenvector of A‘ corresponding 
to the eigenvalue A, of A‘ (and hence of A), the 
eigenvalue A, can be changed to a new real value px 
without altering the other eigenvalues. To achieve 
this, the feedback gain K is chosen as 


32 


Show that the system represented by 


2.0 0 
X(t)=] 0 -1  Ofx(t) +] 1] ud) 
=3 -3 -2 0 


is controllable, and find the eigenvalues and 
corresponding eigenvectors of the system matrix. 
Deduce that the system is unstable in the absence 
of control, and determine a control law that will 
relocate the eigenvalue corresponding to the 
unstable mode at the new value —S. 


A second-order system is modelled by the 
differential equations 


xX, + 2x, -—4x, =u 
X,—X, =u 
coupled with the output equation 
ye 35 
(a) Express the model in state-space form. 


(b) Determine the transfer function of the system 
and show that the system is unstable. 


(c) Show that by using the feedback control law 
u(t) = r(t) — ky(t) 


where k is a scalar gain, the system will be 
stabilized provided k > 2. 


(d) Ifr(t) = H(d), a unit step function, and k > 2 
show that y(t) > 1 as t> © if and only if k= 2 : 


(An extended problem) The transient response 
of a practical control system to a unit step input 
often exhibits damped oscillations before reaching 
steady state. The following properties are some 
of those used to specify the transient response 
characteristics of an underdamped system: 


rise time, the time required for the response 
to rise from 0 to 100% of its final value; 

peak time, the time required for the response 
to reach the first peak of the overshoot; 

settling time, the time required for the response 
curve to reach and stay within a range about 
the final value of size specified by an absolute 
percentage of the final value (usually 2% or 
5%); 

maximum overshoot, the maximum peak 
value of the response measured from unity. 
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Figure 5.66 Feedback control system of Review 
exercise 32. 


Consider the feedback control system of 
Figure 5.66 having both proportional and 
derivative feedback. It is desirable to choose the 
values of the gains K and K, so that the system 
unit step response has a maximum overshoot of 
0.2 and a peak time of 1s. 


(a) Obtain the overall transfer function of the 
closed-loop system. 

(b) Show that the unit step response of the system, 
assuming zero initial conditions, may be 
written in the form 


oe) = ls e "| cos Wat —— sin Wgt 
Wa-#) 


where @, = @,/(1 - &), @2 =K and 
20,€ = 1+ KK;. 


(c) Determine the values of the gains K and K, so 
that the desired characteristics are achieved. 

(d) With these values of K and K,, determine the 
rise time and settling time, comparing both the 
2% and 5% criteria for the latter. 


(An extended problem) The mass M, of the 
mechanical system of Figure 5.67(a) is subjected to 
a harmonic forcing term sin wt. Determine the 
steady-state response of the system. 


Figure 5.67 Vibration absorber of 
Review exercise 33. 
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It is desirable to design a vibration absorber to 
absorb the steady-state oscillations so that in the 
steady state x(t) = 0. To achieve this, a secondary 
system is attached as illustrated in Figure 5.67(b). 


(a) Show that, with an appropriate choice of M, 
and K,, the desired objective may be achieved. 

(b) What is the corresponding steady-state 
motion of the mass M,? 

(c) Comment on the practicality of your design. 


(An extended problem) The electronic amplifier 
of Figure 5.68 has open-loop transfer function G(s) 
with the following characteristics: a low-frequency 
gain of 120dB and simple poles at 1 MHz, 10 MHz 
and 25 MHz. It may be assumed that the amplifier 
is ideal, so that K(1 + KB) = 1/B, where f is 

the feedback gain and K the steady-state gain 
associated with G(s). 


Input 


Figure 5.68 Electronic amplifier of Review 
exercise 34. 


(a) Construct the magnitude versus log frequency 
and phase versus log frequency plots (Bode 
plots) for the open-loop system. 

(b) Determine from the Bode plots whether or 
not the system is stable in the case of unity 
feedback (that is, B = 1). 

(c) Determine the value of B for marginal stability, 
and hence the corresponding value of the 
closed-loop low-frequency gain. 

(d) Feedback is now applied to the amplifier to 
reduce the overall closed-loop gain at low 
frequencies to 100 dB. Determine the gain 
and phase margin corresponding to this 
closed-loop configuration. 

(e) Using the given characteristics, express G(s) 
in the form 


K 


ONS Tse eee ae 


and hence obtain the input—output transfer 
function for the amplifier. 

(f) Write down the characteristic equation for the 
closed-loop system and, using the Routh— 
Hurwitz criterion, reconsider parts (b) and (c). 
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482 THE Z TRANSFORM 


6.1 


Introduction 


In this chapter we focus attention on discrete-(time) processes. With the advent of fast 
and cheap digital computers, there has been renewed emphasis on the analysis and 
design of digital systems, which represent a major class of engineering systems. The 
main thrust of this chapter will be in this direction. However, it is a mistake to believe 
that the mathematical basis of this area of work is of such recent vintage. The first 
comprehensive text in English dealing with difference equations was The Treatise of 
the Calculus of Finite Differences by George Boole and published in 1860. Much of the 
early impetus for the finite calculus was due to the need to carry out interpolation and 
to approximate derivatives and integrals. Later, numerical methods for the solution of 
differential equations were devised, many of which were based on finite difference 
methods, involving the approximation of the derivative terms to produce a difference 
equation. The underlying idea in each case so far discussed is some form of approx- 
imation of an underlying continuous function or continuous-time process. There are 
situations, however, where it is more appropriate to propose a discrete-time model from 
the start. 

Digital systems operate on digital signals, which are usually generated by sampling 
a continuous-time signal, that is a signal defined for every instant of a possibly infinite 
time interval. The sampling process generates a discrete-time signal, defined only at 
the instants when sampling takes place so that a digital sequence is generated. After 
processing by a computer, the output digital signal may be used to construct a new 
continuous-time signal, perhaps by the use of a zero-order hold device, and this in 
turn might be used to control a plant or process. Digital signal processing devices 
have made a major impact in many areas of engineering, as well as in the home. For 
example, compact disc players, which operate using digital technology, offer such 
a significant improvement in reproduction quality that recent years have seen them 
rapidly take over from cassette tape players and vinyl record decks. DVD players 
are taking over from video players and digital radios are setting the standard for 
broadcasting. Both of these are based on digital technology. 

We have seen in Chapter 5 that the Laplace transform was a valuable aid in the 
analysis of continuous-time systems, and in this chapter we develop the z transform, 
which will perform the same task for discrete-time systems. We introduce the transform in 
connection with the solution of difference equations, and later we show how difference 
equations arise as discrete-time system models. 

The chapter includes two engineering applications. The first is on the design of 
digital filters, and highlights one of the major applications of transform methods as 
a design tool. It may be expected that whenever sampling is involved, performance will 
improve as sampling rate is increased. Engineers have found that this is not the full 
story, and the second application deals with some of the problems encountered. This 
leads on to an introduction to the unifying concept of the Y transform, which brings 
together the theories of the Laplace and z transforms. 
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The z transform 


6.2.1 


Since z transforms relate to sequences, we first review the notation associated with 
sequences, which were considered in more detail in Chapter 7 of Modern Engineering 
Mathematics. A finite sequence {x,}5 is an ordered set of n + 1 real or complex 
numbers: 


{xxho a {Xo, X15 Xz, +++ 5 X,} 


Note that the set of numbers is ordered so that position in the sequence is important. 
The position is identified by the position index k, where k is an integer. If the number 
of elements in the set is infinite then this leads to the infinite sequence 


{Xiho = {Xoo X1, Xa... 


When dealing with sampled functions of time ¢, it is necessary to have a means of 
allowing for t < 0. To do this, we allow the sequence of numbers to extend to infinity 
on both sides of the initial position x), and write 


{Xp hor = { see y Ng, Ly, XQ, V1, X25 - + } 


Sequences {x,}"., for which x, = 0 (k < 0) are called causal sequences, by analogy 
with continuous-time causal functions f(t)H(t) defined in Section 5.2.1 as 


0 (t<0) 


t)H(t) = 
4 i (t= 0) 


While for some finite sequences it is possible to specify the sequence by listing all the 
elements of the set, it is normally the case that a sequence is specified by giving a 
formula for its general element x,. 


Definition and notation 


The z transform of a sequence {x,}"., is defined in general as 


°° 


#{x}°.=X(z) = yt (6.1) 


jes & 


whenever the sum exists and where z is a complex variable, as yet undefined. 


The process of taking the z transform of a sequence thus produces a function 
of a complex variable z, whose form depends upon the sequence itself. The symbol 
£ denotes the z-transform operator; when it operates on a sequence {x,} it trans- 
forms the latter into the function X(z) of the complex variable z. It is usual to refer 
to {x,}, X(z) as a z-transform pair, which is sometimes written as {x,} © X(z). 
Note the similarity to obtaining the Laplace transform of a function in Section 5.2.1. 
We shall return to consider the relationship between Laplace and z transforms in 
Section 6.7. 
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Example 6.1 


Solution 


For sequences {x,}*., that are causal, that is 
m=O <= ) 
the z transform given in (6.1) reduces to 


ae (6.2) 


k 
Zz 


Bx 5 =X) => 


In this chapter we shall be concerned with causal sequences, and so the definition 
given in (6.2) will be the one that we shall use henceforth. We shall therefore from now 
on take {x,} to denote {x,}9 . Non-causal sequences, however, are of importance, and 
arise particularly in the field of digital image processing, among others. 


Determine the z transform of the sequence 
{x} = {25 (k= 0) 


From the definition (6.2), 


which we recognize as a geometric series, with common ratio r = 2/z between successive 
terms. The series thus converges for |z| > 2, when 


> (2) =f - Ce 1 
kae0 | - 2/z 1-2/z 


Zz 

k=0 
leading to 

#2} === (|z| > 2) (6.3) 

z-2 

so that 

igh=t2} 

Xz) =— 

aie 


is an example of a z-transform pair. 


From Example 6.1, we see that the z transform of the sequence {2"} exists provided 
that we restrict the complex variable z so that it lies outside the circle |z| = 2 in the 
z plane. From another point of view, the function 


X(z)= 5 (lz|>2) 
g= 2 
may be thought of as a generating function for the sequence {2"t, in the sense that the 


coefficient of z“ in the expansion of X(z) in powers of 1/z generates the kth term of 
the sequence {2}. This can easily be verified, since 
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Solution 
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=o -(1-2)) 
z-2 1-2/z zZ 


and, since |z| > 2, we can expand this as 


-1 2 k 
(1-2) Stele (2) eae (2) tue 
Z Zz Zz Zz 


and we see that the coefficient of z* is indeed 2‘, as expected. 
We can generalize the result (6.3) in an obvious way to determine #{a‘}, the z trans- 
form of the sequence {a‘}, where a is a real or complex constant. At once 


oo k 
ky G2 1 
Aa 2 Toa Cel aD 
so that 
Hap =—— (\z| > lal) (6.4) 
Show that 
ayy=s dal > 2 
. 2z+1 : 
Taking a= -} in (6.4), we have 
a-py= 5 =—2— (el >) 
7a zt 2=(-5) : 
so that 
2 
B= 5 (zl > 2) 


Further z-transform pairs can be obtained from (6.4) by formally differentiating 
with respect to a, which for the moment we regard as a parameter. This gives 


dg hy = gl dat - 2/4) 
f atath= 2 /%et) = 8(-2- 


leading to 


Hi ka} =—=— (|z| > lal) (6.5) 
(z- 


‘ 
In the particular case a = | this gives 


Z 


F{k} = = Qzl2 1) (6.6) 
(z-1 
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Example 6.3 


Solution 


Find the z transform of the sequence 
{2k} = {0, 2, 4, 6, 8,...} 
From (6.6), 


ak} = #{0,1,2,3,...}= 5 £= 2 


Using the definition (6.1), 


oe a 6S eae 
AO AOR TOT at ae ge DG 


Z Z Zz Zz 


so that 


H{2kK} = 2K} = (6.7) 


(z- 1) 


Example 6.3 demonstrates the ‘linearity’ property of the z transform, which we shall 
consider further in Section 6.3.1. 
A sequence of particular importance is the unit pulse or impulse sequence 


{6,} = {1} = {1, 0, 0, ...} 
It follows directly from the definition (6.4) that 
EL 6,3 = (6.8) 


In MATLAB, using the Symbolic Math Toolbox, the z-transform of the sequence 
{x,} is obtained by entering the commands 


syms k z 
ZAC IESIAS (5%) 


As for Laplace transforms (see Section 5.2.2), the answer may be simplified using 
the command simple(ans) and reformatted using the pretty command. Con- 
sidering the sequence {x,} = {2"! of Example 6.1, the commands 


syms k z 
Zions) (2k) 


return 
anes (22 / (1/32°a=1) 
Entering the command 
simple(ans) 
returns 


emeax7// (2=2)) 
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Figure 6.1 Sampling 
of a continuous-time 
signal. 


Example 6.4 


Solution 
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z transforms can be performed in MAPLE using the ztrans function; so the 
commands: 
mibieeiag (2° klk, v) 8 
Sialinvoyll al seni (5) 
return 


ie, 
wo 2 


Sampling: a first introduction 


Sequences are often generated in engineering applications through the sampling of 
continuous-time signals, described by functions f(t) of a continuous-time variable f¢. 
Here we shall not discuss the means by which a signal is sampled, but merely suppose 
this to be possible in idealized form. 


F(kT) y=f 


cnr 
* 


16] T 2T 37 4T ST OT kT t 


Figure 6.1 illustrates the idealized sampling process in which a continuous-time 
signal (f) is sampled instantaneously and perfectly at uniform intervals 7, the sampling 
interval. The idealized sampling process generates the sequence 


{FUT} = {f), A(T), f2T), «-. f(AT), -- 3 


Using the definition (6.1), we can take the z transform of the sequence (6.9) to give 


(6.9) 


at fary} = ¥ LED (6.10) 
0 |OF 


whenever the series converges. This idea is simply demonstrated by an example. 


The signal f(t) = e‘H(t) is sampled at intervals T. What is the z transform of the resulting 
sequence of samples? 


Sampling the causal function f(t) generates the sequence 


{FAT)} = {f0), f(T), f2T), «- f(T), «3 


-T ,-2T ,-3T —nT 
St Ser 6% 4 Oy kb gl pace f 
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Then, using (6.1), 


HE f(kT)} = YS = x (= 
k=0 7 k=0 
so that 
He} =—-=— (\z| > e) 
Z2 =e 


(6.11) 


It is important to note in Example 6.4 that the region of convergence depends on the 


sampling interval 7. 


(2) In MATLAB the commands 


Sivas le Ww 

ztrans (exp (-k*T) ); 

pretty (simple(ans) ) 
return 

aia! = 4 // (4-er90) (<4) )) 
which confirms (6.11). 

In MAPLE the commands: 
ztrans (exp (-k*T) ,k,z); 
Salerno) lestistayaneeop) 

return 


Ts 
aS 


Ny 
Zee oie 


6.2.3 Exercises 
il Calculate the z transform of the following sequences, 2 
stating the region of convergence in each case: 
@ (Do © B45 © 
(d) {-25} (©) {3k} 


—2ot 


The continuous-time signal f(t) = e~°’, where @ is 
a real constant, is sampled when ¢ = 0 at intervals 
T. Write down the general term of the sequence 
of samples, and calculate the z transform of the 
sequence. 


Properties of the z transform 


In this section we establish the basic properties of the z transform that will enable us to 
develop further z-transform pairs, without having to compute them directly using the 


definition. 
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Example 6.5 


Solution 
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The linearity property 


As for Laplace transforms, a fundamental property of the z transform is its linearity, 
which may be stated as follows. 


If {x,} and {y,} are sequences having z transforms X(z) and Y(z) respectively and if 
a and f are any constants, real or complex, then 


Flax, + By} = aF{x,} + BE{y,} = aX(z) + BY(Z) (6.12) 


As a consequence of this property, we say that the z-transform operator & is a linear 
operator. A proof of the property follows readily from the definition (6.4), since 


co 


OX, + Py Xp Vy 
F{ 01, + By,} = FSP - SE py % 
2 k=0 7 k=0 7 


= aX(z) + BY(z) 


The region of existence of the z transform, in the z plane, of the linear sum will be the 
intersection of the regions of existence (that is, the region common to both) of the 
individual z transforms X(z) and Y(z). 


The continuous-time function f(t) = cos wt H(t), @ a constant, is sampled in the ideal- 
ized sense at intervals T to generate the sequence {cos Aw7}. Determine the z transform 
of the sequence. 


Using the result cos kT = }(e"°" + e“°") and the linearity property, we have 
{cos koT} = F{ Leite” + Leitory = 1 Hfeilory 4 Lee hor) 


Using (6.7) and noting that Je”“°"| = |e*°"| = 1 gives 


Z + 1 Zz 
joT 2 -joT 
Z=e 


F{cos k@T} = } (Iz| > 1) 


z-e 
-joT jor. 
z(z-e°” )+2(z-e° 


1 
© GP (or +¢ ez 41 


leading to the z-transform pair 


#fcos koT} = 22-998 @PD yz) > 1) (6.13) 


z -2zcos@T+1 


In a similar manner to Example 6.5, we can verify the z-transform pair 


zsin oT 


¥{sinkoT} = rN 
z -2zcos@T+1 


(Iz| > 1) (6.14) 


and this is left as an exercise for the reader (see Exercise 3). 
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EJ 


6.3.2 


Check that in MATLAB the commands 
syms k 2 1@ 7 
Bice (Cos (Lear) ) 2 
pretty (simple(ans) ) 
return the transform given in (6.13) and that the MAPLE commands: 
Zc eins (COSMO) pI, ) F 
Salmo ll slay? (3) g 


do likewise. 


The first shift property (delaying) 


In this and the next section we introduce two properties relating the z transform of a 
sequence to the z transform of a shifted version of the same sequence. In this section 
we consider a delayed version of the sequence {x,}, denoted by {y,}, with 


Vi = Xk-ky 


Here ky is the number of steps in the delay; for example, if k) = 2 then y, = x,5, 
so that 


Yo =X. Vy FX, V2 =X. V3 =X] 


and so on. Thus the sequence {y,} is simply the sequence {x,} moved backward, or 
delayed, by two steps. From the definition (6.1), 


co 


ava Sy see Fe 
rony= Duy = y =e 
k=0 k=0 p=-k, 
where we have written p = k — ky. If {x,} is a causal sequence, so that x, = 0 (p < 0), 
then 


where X(z) is the z transform of {x;}. 
We therefore have the result 


Bins} = Hae} (6.15) 


which is referred to as the first shift property of z transforms. 

If {x,} represents the sampled form, with uniform sampling interval 7, of the con- 
tinuous signal x(t) then {x,_,,} represents the sampled form of the continuous signal 
x(t — kyT) which, as illustrated in Figure 6.2, is the signal x(t) delayed by a multiple 
ky of the sampling interval 7. The reader will find it of interest to compare this result 
with the results for the Laplace transforms of integrals (5.16). 
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Figure 6.2 
Sequence and its 
shifted form. 


Example 6.6 


Solution 
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{xi} 


O Ff 273T nT t re) koT (ky +0)T t 


The causal sequence {x,} is generated by 
m=G) (k= 0) 


Determine the z transform of the shifted sequence {x,_,}. 


By the first shift property, 
1 k 
F{x,2} = = FL(4)'} 
Zz 


which, on using (6.4), gives 


Z 


1 1 2 2 
Siest = (elo =4-—= 
ZZ-35 


; =-—4— (z)>4 
z2z-1 2(2z-1) 


We can confirm this result by direct use of the definition (6.1). From this, and the fact 
that {x,} is a causal sequence, 


{Xjo ARK 55. X 45 Nyy Xi sa0e f= 40,0, 1, so geeeed 


Thus, 
Hix. = 0+0+t4+te44...=4i+ti ts...) 
z 2z 4z Zz 2z 4z 
1 
Siagere zp) Senn | > 2) 
Ze=5 z(2z-1) 


The second shift property (advancing) 


In this section we seek a relationship between the z transform of an advanced version 
of a sequence and that of the original sequence. First we consider a single-step 
advance. If {y,} is the single-step advanced version of the sequence {x,} then {y,} is 
generated by 


Ve=Xnr (k= 0) 
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6.3.4 


Then 


By }= y= =z) 


k=0 k=0 i=0 ~ 
and putting p =k + | gives 
Ft nowy ag y ay = zX(z) - zx 
Ves = ye = pes = 0 
p=l p=0 


where X(z) is the z transform of {x,}. 
We therefore have the result 


EX} = ZX(Z) — 2X (6.16) 
In a similar manner it is readily shown that for a two-step advanced sequence {x;,,,} 

Boy} = 27 X(z) — 2°Xy — 2X (6.17) 
Note the similarity in structure between (6.16) and (6.17) on the one hand and those for 
the Laplace transforms of first and second derivatives (Section 5.3.1). In general, it is 
readily proved by induction that for a ky-step advanced sequence {X;,,,t 

kg-l 
HA Xirz,} = Z°X(z) - eee (6.18) 


n=0 


In Section 6.5.2 we shall use these results to solve difference equations. 


Some further properties 


In this section we shall state some further useful properties of the z transform, leaving 
their verification to the reader as Exercises 9 and 10. 


(i) Multiplication by a‘ 
If Z{x,} = X(z) then for a constant a 


Ftakx,} = X(a'z) (6.19) 


(ii) Multiplication by k" 
If Z{x,} = X(z) then for a positive integer n 


HK} = (-24) x@ (6.20) 
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Note that in (6.20) the operator —z d/dz means ‘first differentiate with respect 
to z and then multiply by —z’. Raising to the power of m means ‘repeat the 
operation 7 times’. 


(iii) Initial-value theorem 


If {x,} is a sequence with z transform X(z) then the initial-value theorem states 
that 


lim X(z) = Xo (6.21) 


(iv) Final-value theorem 


If {x,} is a sequence with z transform X(z) then the final-value theorem states 
that 


lim x, = lim(1 - z')X(z) (6.22) 


ko 00 z>1 


provided that the poles of (1 — z')X(z) are inside the unit circle. 


6.3.5 Table of z transforms 


It is appropriate at this stage to draw together the results proved so far for easy access. 
This is done in the form of a table in Figure 6.3. 


Figure 6.3 A short 


table of z transforms. {x,} (k = 0) a Region of existence 
i = lee) il Allz 
0 (k>0) 

(unit pulse sequence) 

x, = 1 (unit step sequence) = |e al 
Be 

x, = a‘ (a constant) os z|>|al 
i 

Tels J z|>1 
Cale 

x, = ka (a constant) Z. z|>a 
(z-a)" 

x, = &"” (T constant) Z z|>e7 
Bae 

X, = COSkOT (c, T constants) —2(z = cos WT) Ze 
z -2zc0s oT +1 

x, = SINk@T (@, T constants) —2sin of z>1 


z -2zcos of +1 
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6.3.6 Exercises 


Check your answers using MATLAB or MAPLE whenever possible. 


3 Use the method of Example 6.5 to confirm (6.14), 6 Determine %{(+)*}. Using (6.6), obtain the z 
namely transform of the sequence {k(4)*}. 
7 Show that for a constant a 


F{sin k@T} = zsin oT 


2 -2zcos@Tt+1 


(a) #{sinh kar} = = zsinha 


z -2zcosha+l 


where @ and 7 are constants. 2 
z -zcosha 


(b) F{cosh ka} = ; 
z -2zcosha+1 


4 — Use the first shift property to calculate the z 
transform of the sequence {y,}, with 8 


: 3 
Vk 
k- 


3) 


Sequences are generated by sampling a causal 
continuous-time signal u(t) (tf = 0) at uniform 
intervals 7. Write down an expression for uw, the 
general term of the sequence, and calculate the 
corresponding z transform when u(f) is 


(a) e* (b) sint 


(k < 3) 
(k = 3) 
where {x,} is causal and x, = G )*. Confirm your 


result by direct evaluation of #{y,} using the 
definition of the z transform. 9 


(c) cos 2t 


Prove the initial- and final-value theorems given in 
(6.21) and (6.22). 
5 Determine the z transforms of the sequences 


(a) {(-2)3 (b) {coskr} 


PBA The inverse z transform 


In this section we consider the problem of recovering a causal sequence {x,} from 
knowledge of its z transform X(z). As we shall see, the work on the inversion of Laplace 
transforms in Section 5.2.7 will prove a valuable asset for this task. 


10 Prove the multiplication properties given in (6.19) 
and (6.20). 


Formally the symbol ¥'LX(z)] denotes a causal sequence {x,} whose z transform is 
X(z); that is, 


if #{x,} =X(z) then {x,} = %'[X(z)] 

This correspondence between X(z) and {x,} is called the inverse z transformation, 
{x,} being the inverse transform of X(z), and #' being referred to as the inverse 
z-transform operator. 

As for the Laplace transforms in Section 5.2.8, the most obvious way of finding the 
inverse transform of X(z) is to make use of a table of transforms such as that given in 
Figure 6.3. Sometimes it is possible to write down the inverse transform directly from 
the table, but more often than not it is first necessary to carry out some algebraic manip- 
ulation on X(z). In particular, we frequently need to determine the inverse transform of 
a rational expression of the form P(z)/Q(z), where P(z) and Q(z) are polynomials in z. 
In such cases the procedure, as for Laplace transforms, is first to resolve the expression, 
or a revised form of the expression, into partial fractions and then to use the table of 
transforms. We shall now illustrate the approach through some examples. 
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Example 6.7 


Solution 


Example 6.8 


Solution 
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Inverse techniques 


Find 


From Figure 6.3, we see that z/(z — 2) is a special case of the transform z/(z — a), with 
a= 2. Thus 


1 el ee 
"|| = 424 


Find 
1 Z 
- Fe -1)@- 5 


Guided by our work on Laplace transforms, we might attempt to resolve 


oS 
er eee 


into partial fractions. This approach does produce the correct result, as we shall show 
later. However, we notice that most of the entries in Figure 6.3 contain a factor z in the 
numerator of the transform. We therefore resolve 


a 


Zz (z-1)(z-2) 
into partial fractions, as 


Yzy__l 


= 
Zz z-2 z-1l 
so that 


V3). 2 
ie) z-2 z-l 


Then using the result ¥'[z/(z — a)] = {a‘} together with the linearity property, we have 


Sees |e 7-2 (|e 
= {2)- {1} (k= 0) 


so that 


1 Zz _ gk _ — 
x aoe 1} (k=0) (6.23) 
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EY 


Suppose that in Example 6.8 we had not thought so far ahead and we had simply 
resolved Y(z), rather than Y(z)/z, into partial fractions. Would the result be the same? 
The answer of course is ‘yes’, as we shall now show. Resolving 


—_~ 2 
BY a yaaD) 


into partial fractions gives 


2 1 
Y(z) = — - — 
2 z-2 z-l 


which may be written as 


it follows from the first shift property (6.15) that 


gee | az) ee (k > 0) 
[22 -2| 0 (k = 0) 
Similarly, 
of [L_z_| -{3 ={l} (k>0) 
[22-1 0 (k= 0) 


Combining these last two results, we have 


¥'(¥()] = #' L 25] ~ ¥! E 4 
Zz 


Z-2 zz-1 


a (k > 0) 
0 (k= 0) 


which, as expected, is in agreement with the answer obtained in Example 6.8. 

We can see that adopting this latter approach, while producing the correct result, 
involved extra effort in the use of a shift theorem. When possible, we avoid this by 
‘extracting’ the factor z as in Example 6.8, but of course this is not always possible, 
and recourse may be made to the shift property, as Example 6.9 illustrates. 


The inverse z-transform {x,} of X(z) is returned in MATLAB using the command 


il vattetercliansy (0X (24) 7 s)) 


[Note: The command iztrans(X(z)) by itself returns the inverse transform 


expressed in terms of n rather than k.] 


For the z-transform in Example 6.8 the MATLAB command 


aziciwens (2/ ( (21) (2—2)) )) IR) 
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returns 
ans=-1+2%k 


as required. 
The inverse z-transform can be performed in MAPLE using the invztrans 


function, so that the command 

invgetrens (2/ (2z°2=3)* 242) Zz, ke); 
also returns the answer 

2 = A 


Example 6.9 Find 
1 2z7+1 
ail Se 1)\(z- 71 


Solution In this case there is no factor z available in the numerator, and so we must resolve 


2z+1 


"= GE-3) 


into partial fractions, giving 


1 v4 _ 571) ae 

x ak ly} (k20) 

x! P| ={3'l (k=0) 
z-3 


it follows from the first shift property (6.15) that 


i L al 5 i. (k > 0) 
zz+l 0 (k = 0) 


gilt 2.) Y (k > 0) 
zz-3 0 1 th=0) 
Then, from the linearity property, 


1 1 
¥' lye) =H" ES +19" peel 


zz+l 
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giving 
| —2z41_) _ [LCD +73} (E> 0) 
(z+ 1)(z-3) 0 (k= 0) 
(2) In MATLAB the command 
Ie eeUNS (( (Qe IL) f ( (zee il) (= 3h) }) ,, Ie) 


returns 
einsi== iL // elmer ie inl || (0) ]] (le) iL a (aL) Ses ae 


[Note: The charfcn function is the characteristic function of the set A, and is defined 
to be 


it enisnine 


charfcen[A](k) = 
Ee to if k is not in A 


Thus charfen [0](4) = 1 if &= 0 and 0 otherwise.] 
It is left as an exercise to confirm that the answer provided using MATLAB 
concurs with the calculated answer. 


It is often the case that the rational function P(z)/Q(z) to be inverted has a quadratic 
term in the denominator. Unfortunately, in this case there is nothing resembling the 
first shift theorem of the Laplace transform which, as we saw in Section 5.2.9, proved 
so useful in similar circumstances. Looking at Figure 6.3, the only two transforms with 
quadratic terms in the denominator are those associated with the sequences {cos k@T} 
and {sink@T}. In practice these prove difficult to apply in the inverse form, and a 
‘first principles’ approach is more appropriate. We illustrate this with two examples, 
demonstrating that all that is really required is the ability to handle complex numbers 
at the stage of resolution into partial fractions. 


Example 6.10 _ Invert the z transform 


v4 


2 2 
Z+a 


Y(z) = 


where a is a real constant. 


Solution In view of the factor z in the numerator, we resolve Y(z)/z into partial fractions, giving 


ie 


Zz zt+a? (z+ja)(z-ja) ~ (j2a (z-ja) j2a(z+ja) 


That is 


W(2) = 3 (2 - 2) 


j2a\z-ja ztja 
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Using the result ¥"'[z/(z — a)] = {a"}, we have 


"| = S{Gay}sdi'ay 


Z-Jja 


_ ~ = {(-ja)"} = {(-j)"a‘} 


zt+ja 
From the relation e!® = cos @ + j sin@, we have 
joe? -j=e% 


so that 


=| = | {a\(el™’)'} = fake!!™?} = {a'(cos tkn + jsin tkn)} 


Z-ja 


x" F aA = fa'(cos kn — j sin km) } 


The linearity property then gives 
; 
¥F'[Y(z)] = | Eco kn +jsin +kn — cos $kn +jsin uk 


= {a sin Lk} 


Whilst MATLAB or MAPLE may be used to obtain the inverse z transform when 
complex partial fractions are involved, it is difficult to convert results into a simple 
form, the difficult step being that of expressing complex exponentials in terms of 
trigonometric functions. 


Example 6.11 _—_Invert 
¥(z) = —+— 


z-z+1 


Solution The denominator of the transform may be factorized as 


atte (e-t-iS)e-t8) 


tjn/3 : : 
im", so the denominator may be written as 


In exponential form we have } + ji/3 =e 
2o-zt+la=(z—e™yz— ee?) 


We then have 


Y(@) _ 1 
z (z-e™?)(z-e") 
which can be resolved into partial fractions as 
YQ) _ 1 1 1 1 


jn/3 -jn/3 jn/3 cs -jn/3 jn/3 -jn/3 
Z e -eé Z=6 e -e Z=€ 
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Noting that sin 6 = (e!? — e*)/j2, this reduces to 


Yz)o_ Nz 
z  j2sininmz-e™* j2sin'nz-e""" 
aon ee See ee 2 
igece” jGe2e"" 


Using the result ¥"[z/(z — a)] = {a"}, this gives 


val [Y(c)] = ae 7 gis) _ {2,1 fs kr} 


V 


We conclude this section with two further examples, illustrating the inversion 
technique applied to frequently occurring transform types. 


Example 6.12 _ Find the sequence whose z transform is 


_ 2427 4+1 
~ 3 
Zz 


F(z) 


Solution F(z) is unlike any z transform treated so far in the examples. However, it is readily 
expanded in a power series in z' as 


F(z) =1+244 
4 2 
Using (6.4), it is then apparent that 
Z"LF@)1 = (63 = (1, 2, 0, 1,0, 0, ...} 


(Z) The MATLAB command 
algae se eas! (( (a4 Shae ere se iL )) og 3} Mis) 
returns 
charfcn[0](k)+2*charfcn[1] (k) +charfcn[3](k) 
which corresponds to the sequence 
(ile 2 uOute ON Onset 


Example 6.13 — Find ¥"'[G(z)] where 


Gaye As. 
(z- 1I)(@-e") 


where a and 7 are positive constants. 
Solution Resolving into partial fractions, 


2s eee See aaa 


-aT 
Zz z-l z-e 
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giving 
1 1 


GZ) = “a 
z-e¢ 


z-1 
Using the result ¥"[z/(z — a)] = {a"}, we have 
EG) ={1-e“)} (k= 0) 
In this particular example G(z) is the z transform of a sequence derived by sampling the 
continuous-time signal 
f= 1-2" 


at intervals 7. 


The MATLAB commands 
syms k zaT 
iztrans ((z* (l-exp(-a*T)))/((z-1) * (z-exp(-a*T))),k); 
pretty (simple(ans) ) 
return 
ans=1-exp (-aT)* 
In MAPLE the command 
invztrans ((z*(l-exp(-aT) ))/((z-1) * (z-exp(-aT))),z,k); 


returns 


6.4.2 Exercises 


Confirm your answers using MATLAB or MAPLE whenever possible. 


11 Invert the following z transforms. Give the general (f) Zz és) 22° -7z 
term of the sequence in each case. 2 -2/3z+4 8 (z- 1 (z- 3) 
Zz Zi Zz ‘ 
yo eee (c) z-} (h) ———+_ 
P . : (2-1) (2-241) 
(d) ene (e) oa (f) rene 
; Pe ards 13. “Find ¥'[Y(z)] when Y(z) is given by 
C2 eer Gy a= 
Z= 1 zt+l (a) re (b) 144-2 
12 By first resolving Y(z)/z into partial fractions, find a 4 ee 
¥"[¥(z)] when Y(z) is given b 2 5 
[Y(2)] (2) is g y (c) a (d) 142432 
(a) ——— (b) ——+—_ z z 3z+1 
(z- 1)(z+2) (2z+1)(z- 3) 3 2 2 
2 > (e) 2z +6z +5z+1 (f) 22° = 1247] 
— 02241 -1)'(2-2 
(Qz+D@-1 27 +2-1 ea ety eee) 
(ce) sZ_ [Hint 2 +1 =(2+ (2-3) (g) 3 
z+ go = 322. 
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Discrete-time systems and difference equations 


6.5.1 


Figure 6.4 Discrete- 
time signal processing 
system. 


In Chapter 5 the Laplace transform technique was examined, first as a method for 
solving differential equations, then as a way of characterizing a continuous-time system. 
In fact, much could be deduced concerning the behaviour of the system and its pro- 
perties by examining its transform-domain representation, without looking for specific 
time-domain responses at all. In this section we shall discuss the idea of a linear 
discrete-time system and its model, a difference equation. Later we shall see that the 
z transform plays an analogous role to the Laplace transform for such systems, by 
providing a transform-domain representation of the system. 


Difference equations 


First we illustrate the motivation for studying difference equations by means of an 
example. 

Suppose that a sequence of observations {x,} 1s being recorded and we receive 
observation x, at (time) step or index k. We might attempt to process (for example, 
smooth or filter) this sequence of observations {x,} using the discrete-time feedback 
system illustrated in Figure 6.4. At time step k the observation x, enters the system as 
an input, and, after combination with the ‘feedback’ signal at the summing junction S, 
proceeds to the block labelled D. This block is a unit delay block, and its function is to 
hold its input signal until the ‘clock’ advances one step, to step k + 1. At this time the 
input signal is passed without alteration to become the signal y,,,, the (k + 1)th member 
of the output sequence {y,}. At the same time this signal is fed back through a scaling 
block of amplitude @ to the summing junction S. This process is instantaneous, and at 
S the feedback signal is subtracted from the next input observation x,,,; to provide the 
next input to the delay block D. The process then repeats at each ‘clock’ step. 

To analyse the system, let {r,} denote the sequence of input signals to D; then, 
owing to the delay action of D, we have 


Viet = 
Also, owing to the feedback action, 
=X, — AY, 
where @ is the feedback gain. Combining the two expressions gives 
Vier = X_— Vy 
or 
Veay + OV, = Xp (6.24) 


Equation (6.24) is an example of a first-order difference equation, and it relates adjacent 
members of the sequence {y,} to each other and to the input sequence {x;}. 


{xx} {rx} {yx} 
; | | 
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Figure 6.5 Thesystem 
for Example 6.14. 


Solution 
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A solution of the difference equation (6.24) is a formula for y,, the general term of 
the output sequence {y,}, and this will depend on both & and the input sequence {x,} as 
well as, in this case, the feedback gain a. 


Find a difference equation to represent the system shown in Figure 6.5, having input 
and output sequences {x,} and {y,} respectively, where D is the unit delay block and a 
and b are constant feedback gains. 


Introducing intermediate signal sequences {7,} and {v,} as shown in Figure 6.5, at each 
step the outputs of the delay blocks are 


Yer = (6.25) 

Vig =H (6.26) 
and at the summing junction 

n=X,—av, + by, (6.27) 
From (6.25), 

Vien = Vit 


which on using (6.26) gives 
Vin = 
Substituting for 7, from (6.27) then gives 
Vian = X, — av, + by, 
which on using (6.25) becomes 
Ven = Xp - Wi + bY, 
Rearranging this gives 
Vir2 + Wie — OY, = Xy (6.28) 


as the difference equation representing the system. 


The difference equation (6.28) is an example of a second-order linear constant- 
coefficient difference equation, and there are strong similarities between this and a second- 
order linear constant-coefficient differential equation. It is of second order because the 
term involving the greatest shift of the {y,} sequence is the term in y,,5, implying a shift 
of two steps. As demonstrated by Example 6.14, the degree of shift, or the order of the 
equation, is closely related to the number of delay blocks in the block diagram. 
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6.5.2 


Example 6.15 


Solution 


The solution of difference equations 


Difference equations arise in a variety of ways, sometimes from the direct modelling of 
systems in discrete time or as an approximation to a differential equation describing the 
behaviour of a system modelled as a continuous-time system. We do not discuss this 
further here; rather we restrict ourselves to the technique of solution but examples of 
applications will be apparent from the exercises. The z-transform method is based upon 
the second shift property (Section 6.3.3), and it will quickly emerge as a technique 
almost identical to the Laplace transform method for ordinary differential equations 
introduced in Section 5.3.3. We shall introduce the method by means of an example. 


If in Example 6.14, a = 1, b = 2 and the input sequence {x,} is the unit step sequence 
{1}, solve the resulting difference equation (6.28). 


Substituting for a, b and {x,} in (6.28) leads to the difference equation 

Veot Vier -~2y.=1 (k= 0) (6.29) 
Taking z transforms throughout in (6.29) gives 

ZL YVo2 + Ver — 2} = FA, 1, 1,...} 


which, on using the linearity property and the result Z{1} = z/(z— 1), may be written as 


Eo} + Za} — 2A yi} = 


z-l 


Using (6.16) and (6.17) then gives 


[2°¥@) — zy 291] + 2Y@) - 290] - 2¥(@) = 
a 
which on rearranging leads to 
(227+z-2)¥(z)= 4 +27 yy + 2(y, + Yo) (6.30) 
ae 


To proceed, we need some further information, namely the first and second terms y, and 
y, of the solution sequence {y,}. Without this additional information, we cannot find a 
unique solution. As we saw in Section 5.3.3, this compares with the use of the Laplace 
transform method to solve second-order differential equations, where the values of the 
solution and its first derivative at time ¢ = 0 are required. 

Suppose that we know (or are given) that 


Y = 0, yal 
Then (6.30) becomes 
(2° +2-2)¥(z)=z+ a 
z-1 


or 


(2+ 2)(z-1)¥@)=z+ 
z-1 
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and solving for Y(z) gives 


2 
Zz Zz Zz 


28 (6.31) 
(z+2)(z-1) (z+2)(z-1) (z+2)(z-1) 


To obtain the solution sequence {y,}, we must take the inverse transform in (6.31). 
Proceeding as in Section 6.4, we resolve Y(z)/z into partial fractions as 


C3 ae 


= = 42 2 

z (z+2)(z-1y *(z-1) 7z-1 7242 
and so 

¥(z) =!—+— 22 _2 4 


+ 
(g=1y ?2=1 °2#2 


Using the results ¥"'[z/(z — a)] = {a*} and ¥"[z/(z — 1)°] = {k} from Figure 6.3, we 
obtain 


{yet = {gk +2- 2-2) (k= 0) 


as the solution sequence for the difference equation satisfying the conditions y, = 0 
and y, = 1. 


The method adopted in Example 6.15 is called the z-transform method for solving 
linear constant-coefficient difference equations, and is analogous to the Laplace 
transform method for solving linear constant-coefficient differential equations. 

To conclude this section, two further examples are given to help consolidate under- 
standing of the method. 


Such difference equations can be solved directly in MAPLE using the rsolve 
command. In the current version of the Symbolic Math Toolbox in MATLAB there 
appears to be no equivalent command for directly solving a difference equation. 
However, as we saw in Section 5.5.5, using the maple command in MATLAB 
lets us access MAPLE commands directly. Hence, for the difference equation in 
Example 6.15, using the command 


maple(‘rsolve({y (k+2) +y (k+1) -2*y (k) 
=1,y(0)=0,y(1)=1},y(k))”) 


in MATLAB returns the calculated answer 
-2/9* (-2) *k+2/94+1/3*k 


In MAPLE difference equations can be solved directly using rsolve, so that the 
command 


rsolve({y (k+2) +y (k+1)-2*y (k)=1,y (0)=0,y(1)=1},y(k)); 


returns 
D Qian)" k 
2 8 4 & 
9 9 3 
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Example 6.16 _— Solve the difference equation 
8Vn2 - Vert vy=9 (k= 0) 
given that y) = 1 and y, = 3. 


Solution Taking z transforms 
BZ (Var — OL ar} + Zt = 9Z{1} 
Using (6.16) and (6.17) and the result #{1} = z/(z — 1) gives 


8[z?¥(z) — z*vy — zi] - O[zM@) - 24] + Ye) = = 


z-l 
which on rearranging leads to 
(82? — 62 + 1)¥(z) = 8z2y9 + 8zy, — 6zyy + a 
a 
We are given that y) = 1 and y, = 3, so 
2 = 2 Oz 
(82° — 6z + 1) ¥(z) = 82° + 62+ | 
a 
or 
Y2)___—8z46 9 
e (4821)@2-1) Ge=1)O@z- be-)D 
= : 


ee 
(z-Z)(-5) @-DE-)E-) 
Resolving into partial fractions gives 


a a a ee 


Zz a=5 z-4 Zz i a-5 z-l 
ae ae ee 
Z-4 g=5 z-l1 
and so 
¥(z) = 22 - AZ 4 32 
go> 245 Zl 


4 


Using the result ¥'{z/(z — a)} = {a"} from Figure 6.3, we take inverse transforms, to 
obtain 


fy = {20)°- 40543} (k= 0) 


as the required solution. 


(2) Check that in MATLAB the command 


maple(‘rsolve({8*y (k+2) -—6*y (k+1)+y(k)=9,y(0)=1, 
wl) e372 } 737 We) ) 7) 


returns the calculated answer or alternatively use the command rsolve in MAPLE. 
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Example 6.17 _—_ Solve the difference equation 
Vian + 2y,=0 (k= 0) 


given that y, = 1 and y, = \2. 


Solution Taking z transforms, we have 
[z?¥(z) — z*vy) — zy] + 2¥(z) =0 
and substituting the given values of y) and y, gives 
2°¥(z)— 2° — \2z7+ 2¥(z) =0 
or 
(2° +2)¥(z) =z? + \2z 
Resolving Y(z)/z into partial fractions gives 


Y(z) _ zt+y2 _ z+ ,2 


z 242 (z+j/2)(z-jJ2) 


Following the approach adopted in Example 6.13, we write 


jy2=\2e"™, — —jy2= \2e*” 
¥@) z+,2 _ (+j)/j2 _ = j)/j2 
z (z-2e™)(z- 207") z- 20"? z- 20? 
Thus 


1 : Zz : Zz 
1S i 
|| z- 2"? z-j2e™ 


which on taking inverse transforms gives 


= i +). jkm/2 +) -jkn/2 
ind = 1% [+ ie? (1 - je 
= {2'°(costkn+sintkn)} (k= 0) 


as the required solution. 


The solution in Example 6.17 was found to be a real-valued sequence, and this 
comes as no surprise because the given difference equation and the ‘starting’ values y, 
and y, involved only real numbers. This observation provides a useful check on the 
algebra when complex partial fractions are involved. 
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IS) 


(2) If complex partial fractions are involved then, as was mentioned at the end of Ex- 
ample 6.10, it is difficult to simplify answers when determining inverse z transforms 
using MATLAB. When such partial fractions arise in the solution of difference 
equations use of the command evalc alongside rsolve in MAPLE attempts to 
express complex exponentials in terms of trigonometric functions, leading in most 
cases to simplified answers. 

Considering the difference equation of Example 6.17, using the command 


imkeyouL es (( 7 retsvoull Wass (Ay (2) ce 27 (1) 10), 7 (0) Sal we (( aL) 
a2 (L721) pW) ) YD) 


in MATLAB returns the answer 
(Ade TL fea ye (ee (IL) ye ese (A fl a) (eee (GL 792) )) ke 
whilst using the command 


maple(‘evalc(rsolve({y(k+2)+2*y(k)=0,y(0)=1,y(1) 
a2 (2) hp 7 US) 


returns the answer 


exp (ll/2* logi(2)*k)i cos (1i/2* k*pa) +exollh/ 2 log (2) sk) 
‘ist io (Lf Bele yo)a. ) 


Noting that e'°® = 2 it is readily seen that this corresponds to the calculated 
answer 


2"? (cos;kn + sintkn) 


6.5.3 Exercises 


Check your answers using MATLAB or MAPLE whenever possible. 


Find difference equations representing the discrete- (8) Vir — 2Vpn1 +, = O subject to yy =0, y, = 1 
time systems shown in Figure 6.6. 


(b) Vis2 — 8¥ns1 — 9¥, = 0 subject to yy = 2, y, = 1 

(C) Vu2 + 4,= 0 subject to yy = 0, y, = 1 

(d) 2¥pu2 — SV) — 34, = 0 subject to yy = 3, y, =2 
16 ~—_— Using z-transform methods, solve the following 

difference equations: 

(a) OVa2 + Ve — IV, = 3 subject to y=y, =0 

(b) Veo — Vin + OY, = 5 subject to yy =0, y, = 1 

(C) Viz — Viet + OVn = (5 )" Subject to yo =, = 0 


Figure 6.6 The systems for Exercise 14. () Ysa — 3¥mni + 3yn = 1 subject to Y= 1,1 = 0 
(e) 2Vj.9 — 3Vn1 — 2, = 6n + 1 subject to y= 1, 
y=2 


Using z-transform methods, solve the following 
difference equations: (f) Yaz — 4y, = 3n — 5 subject to yy =, =0 
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A person’s capital at the beginning of, and expenditure 
during, a given year k are denoted by C;, and E£, 
respectively, and satisfy the difference equations 


Cyr = 1.5C, - E 
Ex; = 0.21C, + 0.5E, 


(a) Show that eventually the person’s capital 
grows at 20% per annum. 

(b) Ifthe capital at the beginning of year | is £6000 
and the expenditure during year 1 is £3720 then 
find the year in which the expenditure is a 
minimum and the capital at the beginning of 
that year. 


The dynamics of a discrete-time system are 
determined by the difference equation 


Ver Viv + OV, = Uy 


Determine the response of the system to the unit 
step input 


\° (k < 0) 

up = 

1 (k20) 

given that y) =y, = 1. 

As a first attempt to model the national economy, 


it is assumed that the national income /, at year k 
is given by 


I,=C,+P,+G, 


where C, is the consumer expenditure, P, is private 
investment and G;, is government expenditure. 

It is also assumed that the consumer spending is 
proportional to the national income in the previous 
year, so that 


20 


C,=al,, (0<a<1) 


It is further assumed that private investment is 
proportional to the change in consumer spending 
over the previous year, so that 


P= (C.-C) O<b<S1) 


Show that under these assumptions the national 
income /;, is determined by the difference equation 


Tyg — A + bh + abl, = Gy 


Ifa= 5 ,5=1, government spending is at a constant 
level (that is, G, = G for all A) and J, = 2G, 
I, = 3G, show that 


1, =2[1+ (4) sin  kn]G 
Discuss what happens as k > . 
The difference equation for current in a particular 
ladder network of N loops is 
Rina + Rina 7 i,) + Rina “s ino) = 0 
(0<n<N-2) 


where i, is the current in the (n + 1)th loop, and R, 
and R, are constant resistors. 


(a) Show that this may be written as 
ing — 2 coshai,,,;+1,=0 (0SnS<N-2) 


where 
o: = cosh"! ( + =] 
2R, 
(b) By solving the equation in (a), show that 


. _ i,sinhna- iy sinh(n - 1)a 


n A (2 Saws N) 
sinha 


Naima mecuss characterization 


In this section we examine the concept of a discrete-time linear system and its difference 
equation model. Ideas developed in Chapter 5 for continuous-time system modelling 
will be seen to carry over to discrete-time systems, and we shall see that the z transform 
is the key to the understanding of such systems. 


6.6.1 z transfer functions 


In Section 5.6, when considering continuous-time linear systems modelled by differential 
equations, we introduced the concept of the system (Laplace) transfer function. This is a 
powerful tool in the description of such systems, since it contains all the information 
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on system stability and also provides a method of calculating the response to an 
arbitrary input signal using a convolution integral. In the same way, we can identify a 
z transfer function for a discrete-time linear time-invariant system modelled by a difference 
equation, and we can arrive at results analogous to those of Chapter 5. 

Let us consider the general linear constant-coefficient difference equation model for 
a linear time-invariant system, with input sequence {w,} and output sequence {y,}. Both 
{u,} and {y,} are causal sequences throughout. Such a difference equation model takes 
the form 


Vien ao QV ken-l 7 Dy 2 kn-2 Pee Vx 
= By tresm + Dy Uprm-1 + Dic nee Fee. t bolt, (6.32) 


where k > 0 and n, m (with n = m) are positive integers and the a, and b, are constants. 
The difference equation (6.32) differs in one respect from the examples considered in 
Section 6.5 in that the possibility of delayed terms in the input sequence {u,} is also 
allowed for. The order of the difference equation is n if a,#0, and for the system 
to be physically realizable, n = m. 

Assuming the system to be initially in a quiescent state, we take z transforms 
throughout in (6.32) to give 


(a,Zz" + 4,.2" | +...+ dy) Y(z) = (b,,2" + b,, 2"! +... + by) Uz) 


where Y(z) = #{y,} and U(z) = #{u,}. The system discrete or z transfer function G(z) 
is defined as 


m m-1 
G(z) = a = bie Sha heen (6.33) 
Cas oe ea, 
and is normally rearranged (by dividing numerator and denominator by a,,) so that the 
coefficient of z” in the denominator is 1. In deriving G(z) in this form, we have assumed 
that the system was initially in a quiescent state. This assumption is certainly valid for 
the system (6.32) if 


W=V=---=V1 = 0 


Uj ==... =U = 0 


This is not the end of the story, however, and we shall use the term ‘quiescent’ to mean 
that no non-zero values are stored on the delay elements before the initial time. 
On writing 

P(z) = 6,2" +b, \z" | +... + dy 

O(z) = 4,2" + a, 42" | +...+ ap 
the discrete transfer function may be expressed as 

Gi) = 

O(z) 


As for the continuous model in Section 5.6.1, the equation O(z) = 0 is called the 
characteristic equation of the discrete system, its order, n, determines the order of the 
system, and its roots are referred to as the poles of the discrete transfer function. Like- 
wise, the roots of P(z) = 0 are referred to as the zeros of the discrete transfer function. 
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Solution 


Figure 6.7 

(a) The basic second- 
order block diagram 
substructure; (b) block 
diagram representation 
of (6.34). 


Figure 6.8 (a) The 
z-transform domain 
basic second-order 
block diagram 
substructure; 

(b) the z-transform 
domain block 

diagram representation 
of (6.34). 
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Draw a block diagram to represent the system modelled by the difference equation 


Vera + 3a — Ve = Uy (6.34) 


and find the corresponding z transfer function. 


The difference equation may be thought of as a relationship between adjacent members 


of the solution sequence {y,}. Thus at each time step k we have from (6.34) 
Vor = —3Vienr + Vie + Uy (6.35) 


which provides a formula for y,,, involving y,, y,,; and the input w,. The structure shown 
in Figure 6.7(a) illustrates the generation of the sequence {y,} from {),,.} using two 
delay blocks. 


. {yg42} [>] (yea) [> | {yg] 


(a) (b) 


We now use (6.35) as a prescription for generating the sequence {y,,.} and arrange 
for the correct combination of signals to be formed at each step k at the input summing 
junction S of Figure 6.7(a). This leads to the structure shown in Figure 6.7(b), which is 
the required block diagram. 

We can of course produce a block diagram in the z-transform domain, using a similar 
process. Taking the z transform throughout in (6.34), under the assumption of a quiescent 
initial state, we obtain 

2°¥(z) + 3z¥(z) — Y(z) = U(z) (6.36) 
or 
2°Y(z) =—32z¥(z) + Y(z) + Uz) (6.37) 


The representation (6.37) is the transform-domain version of (6.35), and the z-transform 
domain basic structure corresponding to the time-domain structure of Figure 6.7(a) is 
shown in Figure 6.8(a). 


Spy ¥ Y(2) 


(a) (b) 


The unit delay blocks, labelled D in Figure 6.7(a), become ‘1/z’ elements in the 
z-transform domain diagram, in line with the first shift property (6.15), where a number 
k, of delay steps involves multiplication by z °. 

It is now a simple matter to construct the ‘signal’ transform z’Y(z) from (6.37) and 
arrange for it to be available at the input to the summing junction S in Figure 6.8(a). 
The resulting block diagram is shown in Figure 6.8(b). 
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Example 6.19 


Solution 


The z transfer function follows at once from (6.36) as 


Gz) = B= —1 — (6.38) 
U(z) 2z+3z-1 


A system is specified by its z transfer function 
GG) = . z-1 
z+3z+2 


What is the order 1 of the system? Can it be implemented using only n delay elements? 
Illustrate this. 


If {u,} and {y,} denote respectively the input and output sequences to the system 
then 


Geet a2 | 


U(z) 2 +3z+2 
so that 
(2° + 3z + 2)¥(z) = (z - 1)U(z) 


Taking inverse transforms, we obtain the corresponding difference equation model 
assuming the system is initially in a quiescent state 


Ves2 + 3p + 2Ve = Ups — Ue (6.39) 


The difference equation (6.39) has a more complex right-hand side than the difference 
equation (6.34) considered in Example 6.18. This results from the existence of z 
terms in the numerator of the transfer function. By definition, the order of the 
difference equation (6.39) is still 2. However, realization of the system with two 
delay blocks is not immediately apparent, although this can be achieved, as we shall 
now illustrate. 

Introduce a new signal sequence {r,} such that 


(z° + 3z + 2)R(z) = Uz) (6.40) 


where R(z) = #{r;,}. In other words, {7,} is the output of the system having transfer 
function 1/(z* + 3z + 2). 
Multiplying both sides of (6.40) by z, we obtain 


2(z° + 3z + 2)R(z) = zU(z) 
or 

(z? + 3z + 2)zR(z) = zU(z) (6.41) 
Subtracting (6.40) from (6.41) we have 

(2? + 3z + 2)zR(z) — (2? + 3z + 2)R(z) = ZU(z) — U(z) 
giving 


(z° + 3z + 2)[zR(z) — R(z)] = (2 — U2) 
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Figure 6.9 The z-transform block diagrams for (a) the system (6.40), (b) the system (6.39), and (c) the time-domain 
realization of the system in Example 6.19. 


Example 6.20 


Solution 


Finally, choosing 
Y(z) = zR(z) — R@) (6.42) 
(z° + 3z + 2)¥(z) = (z- 1)U(z) 


which is a realization of the given transfer function. 

To construct a block diagram realization of the system, we first construct a block 
diagram representation of (6.40) as in Figure 6.9(a). We now ‘tap off’ appropriate 
signals to generate Y(z) according to (6.42) to construct a block diagram representation 
of the specified system. The resulting block diagram is shown in Figure 6.9(b). 

In order to implement the system, we must exhibit a physically realizable time-domain 
structure, that is one containing only D elements. Clearly, since Figure 6.9(b) contains 
only ‘1/z’ blocks, we can immediately produce a realizable time-domain structure as 
shown in Figure 6.9(c), where, as before, D is the unit delay block. 


A system is specified by its z transfer function 


4 


G(z) = a 
z +0.3z+0.02 


Draw a block diagram to illustrate a time-domain realization of the system. Find a 
second structure that also implements the system. 


We know that if #{u,} = U{z} and #{ y,} = Y(z) are the z transforms of the input and 
output sequences respectively then, by definition, 


64322 22s (6.43) 


U(z) 2 +0.3z+0.02 
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Figure 6.10 (a) The z-transform block diagram for the system of Example 6.20; and (b) the time-domain 
implementation of (a). 


which may be rewritten as 
(z* + 0.3z + 0.02) ¥(z) = zU(z) 


Noting the presence of the factor z on the right-hand side, we follow the procedure of 
Example 6.19 and consider the system 


(z* + 0.3z + 0.02)R(z) = U(z) (6.44) 
Multiplying both sides by z, we have 
(z* + 0.32 + 0.02)zR(z) = zU(z) 


and so, if the output Y(z) = zR(z) is extracted from the block diagram corresponding to 
(6.44), we have the block diagram representation of the given system (6.43). This is 
illustrated in Figure 6.10(a), with the corresponding time-domain implementation 
shown in Figure 6.10(b). 

To discover a second form of time-domain implementation, note that 


4 2 1 


2 = 
z+03z+0.02 24+0.2 2+0.1 


We may therefore write 


pc aa E = 2 2Z a i)U@) 
so that 
Y(z) = R,(z) — RZ) 
where 
2 
R\(z) = 500 U(z) (6.45a) 
iil 
R,(z) = ere, U(z) (6.45b) 


From (6.45a), we have 


(z + 0.2)R,(z) = 2U(z) 
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Figure 6.11 The block 
diagrams for (a) the 
subsystem (6.45a), 

(b) the subsystem 
(6.45b), and (c) an 
alternative z-transform 
block diagram for 

the system of 
Example 6.20. 
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which can be represented by the block diagram shown in Figure 6.11(a). Likewise, 
(6.45b) may be represented by the block diagram shown in Figure 6.11(b). 

Recalling that Y(z) = R,(z) — R,(2z), it is clear that the given system can be represented 
and then implemented by an obvious coupling of the two subsystems represented by 
(6.45a, b). The resulting z-transform block diagram is shown in Figure 6.11(c). The 
time-domain version is readily obtained by replacing the ‘1/z’ blocks by D and the 
transforms U(z) and Y(z) by their corresponding sequences {u,} and { y,} respectively. 


The impulse response 


In Example 6.20 we saw that two quite different realizations were possible for the 
same transfer function G(z), and others are possible. Whichever realization of the 
transfer function is chosen, however, when presented with the same input sequence 
{u,}, the same output sequence { y,} will be produced. Thus we identify the system as 
characterized by its transfer function as the key concept, rather than any particular 
implementation. This idea is reinforced when we consider the impulse response sequence 
for a discrete-time linear time-invariant system, and its role in convolution sums. 
Consider the sequence 


{5,} = {1, 0,0, ...} 


that is, the sequence consisting of a single ‘pulse’ at k= 0, followed by a train of zeros. 
As we saw in Section 6.2.1, the z transform of this sequence is easily found from the 
definition (6.1) as 


FG, =1 (6.46) 


The sequence {6,} is called the impulse sequence, by analogy with the continuous- 
time counterpart 6(f), the impulse function. The analogy is perhaps clearer on con- 
sidering the transformed version (6.46). In continuous-time analysis, using Laplace 
transform methods, we observed that #{6(t)} = 1, and (6.46) shows that the ‘entity’ 
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Example 6.21 


Solution 


THE Z TRANSFORM 


with z transform equal to unity is the sequence {6,}. It is in fact the property that 
¥{6,} = 1 that makes the impulse sequence of such great importance. 

Consider a system with transfer function G(z), so that the z transform Y(z) of the 
output sequence { y,} corresponding to an input sequence {u,} with z transform U(z) is 


¥(z) = GU) (6.47) 


If the input sequence {y,} is the impulse sequence {6,} and the system is initially 
quiescent, then the output sequence {y5,} is called the impulse response of the system. 
Hence 


Ff Vet = Ys(z) = G@) 


That is, the z transfer function of the system is the z transform of the impulse response. 
Alternatively, we can say that the impulse response of a system is the inverse z trans- 
form of the system transfer function. This compares with the definition of the impulse 
response for continuous systems given in Section 5.6.3. 

Substituting (6.48) into (6.47), we have 


¥(z) = Ys(z)U(Z) 


(6.48) 


(6.49) 


Thus the z transform of the system output in response to any input sequence {z,} is the 
product of the transform of the input sequence with the transform of the system impulse 
response. The result (6.49) shows the underlying relationship between the concepts of 
impulse response and transfer function, and explains why the impulse response (or the 
transfer function) is thought of as characterizing a system. In simple terms, if either of 
these is known then we have all the information about the system for any analysis we 
may wish to do. 


Find the impulse response of the system with z transfer function 


G(z) == 
z+3z4+2 
Using (6.48), 
¥;(z) = =——— = ; 


243242 (2+2)(zt1) 
Resolving Y;(z)/z into partial fractions gives 


Y5(z) _ 1 eee Ceres 
Zz (z+2)(z+1) z+1 24+2 


which on inversion gives the impulse response sequence 


z+1 24+2 


{¥5,} = aa : 


= {(-1)"-(-2)'} (k= 0) 
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Since the impulse response of a system is the inverse z transform of its transfer func- 
tion G(z) it can be obtained in MATLAB using the command 


syms k z 
ii ieenaus) (Ea) , Is) 


so for the G(z) of Example 6.21 


syms k z 
aie eelins (x // (BOP Bm te2)) Ie) 


returns 
aise (=i) k= (=2) Ms 
A plot of the impulse response is obtained using the commands 
mate (724 yA) e 
G=G(z); 
impulse (G) 


Likewise in MAPLE the command 
invztrans (z/ (z*2+3*z+2),z,k); 


returns the same answer 
(-1)* -— (-2)* 


A system has the impulse response sequence 
{ys,}= {a — 0.5*} 


where a > 0 is a real constant. What is the nature of this response when (a) a = 0.4, 
(b) a= 1.2? Find the step response of the system in both cases. 


When a = 0.4 
{¥s,t = {0.4*—0.5*+ 


and, since both 0.4‘ > 0 as k > ~ and 0.5‘ > 0 as k 4 ~, we see that the terms of the 
impulse response sequence go to zero as k > ©, 

On the other hand, when a = 1.2, since (1.2) > © as k > ©, we see that in this case 
the impulse response sequence terms become unbounded, implying that the system 
“blows up’. 

In order to calculate the step response, we first determine the system transfer function 
G(z), using (6.48), as 


G(z) = Yz) = Fak — 0.5" 


giving 


(ele 
7 z-a z-0.5 


The system step response is the system response to the unit step sequence {h,} = 
{1, 1, 1,... } which, from Figure 6.3, has z transform 
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Ff hy} = — 
z-l 
Hence, from (6.46), the step response is determined by 


¥(z) = G(z)F{ hy} = (+ : a Zz 


z-a z-0.5/z-1 
so that 
a sh ee 
Zz (z-a)(z-1) (z-0.5)(z- 1) 
Per: eee ea +(24 1 } 1 
a-lz-a z-0.5 l-a/z-l 
giving 
¥(z) = #42. --2 +(24 | me 
a-lz-a z-0.5 l-a/z-l 
which on taking inverse transforms gives the step response as 
Gia a‘- (0.5)'+(-2+ u } (6.50) 
a-1l l-a 


Considering the output sequence (6.50), we see that when a = 0.4, since (0.4) > 0 
as k — oo (and (0.5)* + 0 as k > 0), the output sequence terms tend to the constant 
value 


L . = 039333 


—2+ = 
1- 0.4 


In the case of a = 1.2, since (1.2) > 0 as k ©, the output sequence is unbounded, 
and again the system ‘blows up’. 


6.6.3 Stability 


Example 6.22 illustrated the concept of system stability for discrete systems. When 
a = 0.4, the impulse response decayed to zero with increasing k, and we observed 
that the step response remained bounded (in fact, the terms of the sequence 
approached a constant limiting value). However, when a = 1.2, the impulse response 
became unbounded, and we observed that the step response also increased without 
limit. In fact, as we saw for continuous systems in Section 5.6.3, a linear constant- 
coefficient discrete-time system is stable provided that its impulse response goes to zero 
as t > oe. As for the continuous case, we can relate this definition to the poles of the 
system transfer function 


_ P@) 
G(z) = 
= 5G) 


As we saw in Section 6.6.1, the system poles are determined as the n roots of its charac- 
teristic equation 


O(z) =a4,2Z" +a, 2" | +...+a)=0 (6.51) 
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For instance, in Example 6.19 we considered a system with transfer function 

G(z) = oo 

z+3z+2 

having poles determined by z* + 3z + 2 =0, that is poles at z=—1 and z=—2. Since the 
impulse response is the inverse transform of G(z), we expect this system to ‘blow up’ 
or, rather, be unstable, because its impulse response sequence would be expected to 
contain terms of the form (—1)‘ and (—2)*, neither of which goes to zero as k > o. 
(Note that the term in (—1)* neither blows up nor goes to zero, simply alternating 
between +1 and —1; however, (—2)* certainly becomes unbounded as k > -.) On 
the other hand, in Example 6.20 we encountered a system with transfer function 


Zz 


G(z) = Peete anata 
z +0.3z4+0.02 


having poles determined by 
O(z) =z? + 0.32 + 0.02 = (z + 0.2)(z + 0.1) =0 


that is poles at z = —0.2 and z = —0.1. Clearly, this system is stable, since its impulse 
response contains terms in (—0.2)‘ and (—0.1)*, both of which go to zero as k > ©», 

Both of these illustrative examples gave rise to characteristic polynomials Q(z) 
that were quadratic in form and that had real coefficients. More generally, O(z) = 0 
gives rise to a polynomial equation of order n, with real coefficients. From the theory 
of polynomial equations, we know that O(z) = 0 has n roots a; (i= 1, 2, ...,), which 
may be real or complex (with complex roots occurring in conjugate pairs). 

Hence the characteristic equation may be written in the form 


O(z) = a,(z — 0,)(z -— 0) ... (z- ,) =0 (6.52) 


The system poles a; (i= 1, 2, ... , ) determined by (6.52) may be expressed in the polar 
form 


a=re* (@=1,2,...,n) 


where 0; = 0 or x if a; is real. From the interpretation of the impulse response as the 
inverse transform of the transfer function G(z) = P(z)/O(2), it follows that the impulse 
response sequence of the system will contain terms in 


jkO, 
a 


k jkO, ke k jkO 
ree. re & ag fe 


Since, for stability, terms in the impulse response sequence must tend to zero as 
k > , it follows that a system having characteristic equation Q(z) = 0 will be stable 
provided that 


r,<1 for i=1,2,...,n 


Therefore a linear constant-coefficient discrete-time system with transfer function 
G(z) is stable if and only if all the poles of G(z) lie within the unit circle |z| < 1 in 
the complex z plane, as illustrated in Figure 6.12. If one or more poles lie outside 
this unit circle then the system will be unstable. If one or more distinct poles lie on 
the unit circle |z| = 1, with all the other poles inside, then the system is said to be 
marginally stable. 
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Figure 6.12 Region of 
stability in the z plane. 


Example 6.23 


Solution 


Im (z) 


J Unit circle |z|=1 


> 
1 Re(z) 


Which of the following systems, specified by their transfer function G(z), are stable? 


(a) G(z) = (b) G(z) = ——.__ (cc) Giz) = a 
703 z-z+0.5 ge = 32 4057-1 


(a) The single pole is at z=—0.25, sor, = 0.25 < 1, and the system is stable. 
(b) The system poles are determined by 
2? -7z+0.5=[z-0.5(1 + j)][z- 0.51 —j)] =0 


giving the poles as the conjugate pair z, = 0.5(1 + j), z, = 0.5(1 — j). The ampli- 
tudes r, =r, = 0.707 < 1, and again the system is stable. 


(c) The system poles are determined by 
z3—327+2.5z-1=(z-2)[z-0.5(1 + j)][z — 0.511 — j)] 


giving the poles as z, = 2, z, = 0.5(1 +j), z,; = 0.5(1 — J), and so their amplitudes 
are r, = 2, r, =1r; = 0.707. Since r, > 1, it follows that the system is unstable. 


According to our definition, it follows that to prove stability we must show that all 
the roots of the characteristic equation 


Oz) =z" +4, 2" | +...+a),=0 (6.53) 


lie within the unit circle |z| = 1 (note that for convenience we have arranged for the 
coefficient of z” to be unity in (6.53) ). Many mathematical criteria have been developed 
to test for this property. One such method, widely used in practice, is the Jury stability 
criterion introduced by E. I. Jury in 1963. This procedure gives necessary and suffi- 
cient conditions for the polynomial equation (6.53) to have all its roots inside the unit 
circle |z| = 1. 

The first step in the procedure is to set up a table as in Figure 6.13 using information 
from the given polynomial equation (6.53) and where 


1 ay bo Dyk Co Cn-2-k 
by = > Cy = ’ d;, > ’ 
Ap An-K by b, Cn-2 Ck 
Yo 12 
ty = 
rr. Yo 
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polynomial equation 
(6.53). 
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Solution 
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Row gr gt gi a 3 z z 
1 1 Byes Oy Qy-k a a, ay 
2 ay a, a a, An-2 a1 1 
3 A, =} b, b, by by-2 by-1 

4 Dy Dy Dis Dyk b, bo 

5 Ay =o Cy Cy CK loner 

6 Cn-2 Cn-3 Cn-4 Cn-2-k Co 

fi A; = dy d, d, dy 

8 ds d,-4 dys dy3-k 

2n-5 AvEr=157, Sy S> 53 

2n-4 S3 So Sy So 

2n-3 A,2=7 r Tp 

2n—2 T> r To 

2n-1 Nowra 


Note that the elements of row 2j + 2 consist of the elements of row 27 + 1 written in the 
reverse order for j= 0, 1, 2,...,; that is, the elements of the even rows consist of the 
elements of the odd rows written in reverse order. Necessary and sufficient conditions 
for the polynomial equation (6.53) to have all its roots inside the unit circle |z| = | are 
then given by 


@ @0)>90, CD"'QCl>0 


(6.54) 
(ii) A, > 0, A; = 0, A; ea 0, ar) A, > 0, A 


>0 


n—| 


Show that all the roots of the polynomial equation 
FQ) =2) + 32° 42-75 =0 


lie within the unit circle |z| = 1. 


The corresponding Jury stability table is shown in Figure 6.14. In this case 


(i) FU)=14+!-!-L>0 


3 4 12 


COPED =ClCl fi +7=2)>0 


+ 143 1432 4 
Gi) A=Wm>0, A=Ga) -F7> 0 


Thus, by the criteria (6.54), all the roots lie within the unit circle. In this case this is 
readily confirmed, since the polynomial F(z) may be factorized as 


F(z) = (2 — $)(@+ $)(z+ }) =0 


So the roots are z, = 4, z= —} and z,=— 


ul 
2 3° 
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Figure 6.14 Jury 
stability table for 
Example 6.24. 


Example 6.25 


Figure 6.15 Jury 
stability table for 
Example 6.25. 


Solution 


Row z 


z z! z 
1 al aL 
1 | Ei 4 12 
=p =! L 
2 12 4 3 1 
1 “4 1 -} 1 
3 A, = 5 
1 1 1 ei 
12 l 12 3 12 
= 1B =i =-2 
144 16 9 
-2 =e 143 
4 9 16 144 
M4302 
144 9 
Bs) A, = 
2 143 
9 1 
= 0.936 78 


The Jury stability table may also be used to determine how many roots of the 
polynomial equation (6.53) lie outside the unit circle. The number of such roots is 


determined by the number of changes in sign in the sequence 


Il. Ay Ag aden A 


n—| 


Show that the polynomial equation 
F(z) = 2° - 32? - sz+2=0 


has roots that lie outside the unit circle |z| = 1. 


Determine how many such roots there are. 


Row 2 z? z Zz 
1 a 

1 1 3 ail 3 
2 3 = 4 1 

aoe 45 

45 

4 2 -2 2 
5 A, = -+ 


The corresponding Jury stability table is shown in Figure 6.15. Hence, in this case 


F@)=1-3-14+3=-3 


(-1"FC1) = (C181 -3 + +3) =3 


As F(1) < 0, it follows from (6.54) that the polynomial equation has roots outside the 
unit circle |z| = 1. From Figure 6.15, the sequence 1, A,, A, is 1, Z, -8, and since 
there is only one sign change in the sequence, it follows that one root lies outside the 


unit circle. Again this is readily confirmed, since F(z) may be factorized as 


F(z) = (z-$)(z+}$)(z- 3) =0 


showing that there is indeed one root outside the unit circle at z = 3. 
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Example 6.26 — Consider the discrete-time feedback system of Figure 6.16, for which T is the sampling 
period and & > 0 is a constant gain: 


(a) 


Solution (a) 


(b) 


Figure 6.16 
Discrete-time system 


1 4, 
of Example 6.26. A ,| _« = 
- ' Sampler s(s + 1) ' 


Determine the z transform G(z) corresponding to the Laplace transform G(s). 


Determine the characteristic equation of the system when 7 = 1 and k = 6 and 
show that the discrete-time system is unstable. 


For T= 1 show that the system is stable if and only if 0 < k < 4.33. 


Removing the sampler show that the corresponding continuous-time feedback 
system is stable for all k > 0. 


First invert the Laplace transform to give the corresponding time-domain func- 
tion f(t) and then determine the z transform of f(d): 


k k_ ek 
G =e = - - 
(s) s(st+l) os stl 


f(t)=k- ket 
Gz) = Zk} - Z{kerty = AE - AE = _ke(1 =e) 


me = DG=84 
With k=6 and T= 1 
-1 
G,2) = 6(l-e )z 


(z- 1)(z-e"') 
The closed loop transfer function is 
Ga(Z) 
1+ G,(z) 


giving the characteristic equation 


1+ G,(z) = 0 as (z- 1)(z-e') + 6(1 —e")z=0 
or 

z°+2[6(1-e!)-(l+e]+e7=0 
which reduces to 

z° + 2.3242 + 0.368 = 0 


The roots of this characteristic equation are z, = —0.171 and z, = —2.153. Since 
one of the roots lies outside the unit circle | z| = 1 the system is unstable. 


= G(s) 
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(c) For 7=1 and general gain k > 0 the characteristic equation of the system is 
F(2)=(2-le-e')+k(1-e)z=0 
which reduces to 
F(z) =z? + (0.632k — 1.368)z + 0.368 = 0 
By Jury’s procedure conditions for stability are: 


F(1) = 1 + (0.632k — 1.368) + 0.368 > 0 since k > 0 
2.736 


(-1)°F(-1) = 2.736 — 0.632k > 0 provided k < 22 = 4.33 
0.632 
he | 1 ea 7 
0.568 1 


Thus F(1) > 0, (-1)°F(-1) > 0 and A, > 0 and system stable if and only if k < 4.33. 


(d) In the absence of the sampler the characteristic equation of the continuous-time 
feedback system is | + G(s) = 0, which reduces to 


stst+k=0 


All the roots are in the negative half of the s-plane, and the system is stable, for 
all k > 0. 


Convolution 


Here we shall briefly extend the concept of convolution introduced in Section 5.6.6 to 
discrete-time systems. From (6.45), for an initially quiescent system with an impulse 
response sequence {ys} with z transform Y,(z), the z transform Y(z) of the output 
sequence {y,} in response to an input sequence {u,} with z transform U(z) is given by 


Y(z) = Y«z)U(z) (6.49) 


For the purposes of solving a particular problem, the best approach to determining {y,} 
for a given {u,} is to invert the right-hand side of (6.49) as an ordinary z transform with 
no particular thought as to its structure. However, to understand more of the theory of 
linear systems in discrete time, it is worth exploring the general situation a little further. 
To do this, we revert to the time domain. 

Suppose that a linear discrete-time time-invariant system has impulse response 
sequence {y5,}, and suppose that we wish to find the system response {),} to an input 
sequence {u,}, with the system initially in a quiescent state. First we express the 
input sequence 


{uj} = (Up, Uy, Ua) 6. Uns oF (6.55) 
as 
{u,} = uy{O,} + uj {0.4} + {Ono} +... +u,{O,,,} +... (6.56) 
where 
5. = F (ke) 
1 (k=j) 
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Example 6.27 


In other words, {6,_;} is simply an impulse sequence with the pulse shifted to k = /. 
Thus, in going from (6.55) to (6.56), we have decomposed the input sequence {u;,} 
into a weighted sum of shifted impulse sequences. Under the assumption of an ini- 
tially quiescent system, linearity allows us to express the response {y,} to the input 
sequence {uw,} as the appropriately weighted sum of shifted impulse responses. Thus, 
since the impulse response is {ys}, the response to the shifted impulse sequence 
{6,;} will be {¥5,_,$> and the response to the weighted impulse sequence u,{6,_;} 
will be simply u;{ Vogt Summing the contributions from all the sequences in (6.56), 
we obtain 


Ly = ¥ ys,,} (6.57) 


as the response of the system to the input sequence {u,}. Expanding (6.57), we have 


{yi} = Mol Ys} + mtys, b+... + UAYS,_/t +... 


= Uy {¥s,> Vs Vey 0+ > YS, tee } 
+u,{0, Y5y? V5 eer VS, 1? aces } 
FUAU Oy Vessseg  Vizsnvee } 
+ u,{0, 0, 0, soe 0; V5y> V5,> pa } 
t 
AP ices hth position 


From this expansion, we find that the Ath term of the output sequence is deter- 
mined by 


Vn = » UjYS,,_, (6.58) 
j=0 
That is, 
k 
{yi} = » ws.) (6.59) 
j=0 


The expression (6.58) is called the convolution sum, and the result (6.59) is analogous 
to (5.83) for continuous systems. 
A system has z transfer function 

G2) => 

a 5 


What is the system step response? Verify the result using (6.59). 
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Solution 


From (6.46), the system step response is 
Y(z) = Gl2)Z {hy} 
where {/,} = {1, 1, 1,... }. From Figure 6.3, #{h,} =z/(z — 1), so 


Z 


Y(z)=— 
@) z+3z-1 


Resolving Y(z)/z into partial fractions gives 


Y@) 2 dy dl 

Zz (z+5)(z-1) z-1 Z+5 
so 

Vio) a 2 el 

@) 37-1 *z+t 


Taking inverse transforms then gives the step response as 
{yb = 434 
Using (6.59), we first have to find the impulse response, which, from (6.48), is given by 


1 ees 
{y5,} = FAG] = % Fy 


so that 
{ys} = {(-)'} 


Taking {u,} to be the unit step sequence {h,}, where h, = 1 (k = 0), the step response 
may then be determined from (6.59) as 


pie >» sy.) : » I. cy" 


j=0 


forge} -farger 


Recognizing the sum as the sum to 4+ | terms of a geometric series with common ratio 
—2, we have 


ty} = {cp ca ={2(-1*+42)} = 24 (-D4 


which concurs with the sequence obtained by direct evaluation. 


Example 6.27 reinforces the remark made earlier that the easiest approach to 
obtaining the response is by direct inversion of (6.32). However, (6.59), together with 
the argument leading to it, provides a great deal of insight into the way in which the 
response sequence {y,} is generated. It also serves as a useful ‘closed form’ for the 
output of the system, and readers should consult specialist texts on signals and systems 
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for a full discussion (P. Kraniauskas, Transforms in Signals and Systems, Addison- 
Wesley, Wokingham, 1992). 

The astute reader will recall that we commenced this section by suggesting that we 
were about to study the implications of the input-output relationship (6.49), namely 


Y(z) = Y;(z)U(z) 


We have in fact explored the time-domain input-output relationship for a linear system, 
and we now proceed to link this approach with our work in the transform domain. By 
definition, 


U(z) = Yi mz =ujtt+2 ++ 
k=0 Zz Zz Zz 
Y(z) = }y5,z = yp te ee, ine 
k=0 ie a 
so 
1 1 
TAZ) = toys, + Cova, + Miys,)— + Mays, + Ma, + ava, Fs (6-00) 
Zz 


Considering the kth term of (6.60), we see that the coefficient of z“ is simply 


‘ 
BD My, 


j=0 


However, by definition, since Y(z) = Y;(z)U(z), this is also y(k), the Ath term of the 
output sequence, so that the latter is 


{ye} = {> Hf 


j=0 


as found in (6.59). We have thus shown that the time-domain and transform-domain 
approaches are equivalent, and, in passing, we have established the z transform of the 
convolution sum as 


ay wn] = U(z)V(z) (6.61) 


where 
F{u,} = UZ), F{v,} = V2) 
Putting p = k —7 in (6.61) shows that 


k k 
> UjV,-; = > UjpVp (6.62) 


confirming that the convolution process is commutative. 
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6.6.5 Exercises 


Check your answers using MATLAB or MAPLE whenever possible. 


Zl 


22 


23 


24 


25 


26 


Find the transfer functions of each of the following 27 Following the same procedure as in Example 6.26 

discrete-time systems, given that the system is show that the closed-loop discrete-time system of 

initially in a quiescent state: Figure 6.17, in which k > 0 and t > 0, is stable if 
and only if 


(8) Veer — 3¥ 1 + 2Vp = Uy 


T 
(b) Veo — 3V 1 + 2V6 = Ue — Ue Oe ae coth(;- ) 


(C) Vas — Vier + 2a +e = Ue + Ui 


T 
Draw a block diagram representing the discrete- oe k 
time system + sampler “Is(as +1) 
Via + OSV) + O.25y, = Uy ~ ees 
Hence find a block diagram representation of the 
system 
Via + OSV) + O.25y, = Uy — 0.6141 Figure 6.17 Discrete-time system of Exercise 27. 


Find the impulse response for the systems with 


z transfer function . . 
28 A sampled data system described by the difference 


2 


ti 
@ = Zz b) = Zz equation 

8z+6z+1 z-3z+3 

Yn+1 — Vn = Uy 
z 52° - 12z 

= [ae is controlled by making the input w, proportional to 

zee 008 eee et the previous error according to 
Obtain the impulse response for the systems of 1 
Exercises 21(a, b). uy = KS - Yn) 


where K is a positive gain. Determine the range of 


Which of the following systems are stable? , ; : 
values of K for which the system is stable. Taking 


(8) Wasa + War + 2p = Uy K = 2, determine the response of the system given 
9 P 
=y,=0. 
(b) Wiss = 3¥ 1 = 2, = Uy jas 
(C) 2Ve2 — 2Vier +e = Ups — Ue 29 Show that the system 
(d) 2Vp.2 + 3V 41 — Ve = Ue Vn + Wey + 2Y, = Une (n = 0) 
(©) AY a2 — 3Ven Ye = Mar — 20 has transfer function 
Use the method of Example 6.27 to calculate D(z) = Zz 
the step response of the system with transfer ae 242242 
function 
Show that the poles of the system are at z=—1 +j 
ae and z = —1 —j. Hence show that the impulse 
i oes 
Z-5 response of the system is given by 
Verify the result by direct calculation. h, = £"'D(z) = 2"? sin pnt 
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The relationship between Laplace and z transforms 


Figure 6.18 Sampled 
function /(#). 


Throughout this chapter we have attempted to highlight similarities, where they occur, 
between results in Laplace transform theory and those for z transforms. In this section 
we take a closer look at the relationship between the two transforms. In Section 6.2.2 
we introduced the idea of sampling a continuous-time signal f(‘) instantaneously at 
uniform intervals T to produce the sequence 


{fal )} = {fO), fT), f2T), --. f(T), «3 (6.63) 


An alternative way of representing the sampled function is to define the continuous- 
time sampled version of f(t) as f(t) where 


fH= ¥ f= n7) =¥. flnT)8(0-n7) (6.64) 


n=0 


The representation (6.64) may be interpreted as defining a row of impulses located at 
the sampling points and weighted by the appropriate sampled values (as illustrated in 
Figure 6.18). Taking the Laplace transform of f(t), following the results of Section 5.5.10, 
we have 


v0 =| Syanar-en e” dt 


0 | k=0 


co 


= 5 un | O(t-kT)e™ Ot 


giving 
Lt fior=¥ fenye" (6.65) 


Making the change of variable z = e*” in (6.65) leads to the result 


LAO) =, AUT) 2 = Fle) (6.66) 


k=0 


SOA y=fO 


-- 
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6.8.1 


where, as in (6.10), F(z) denotes the z transform of the sequence {/(AT)}. We can 
therefore view the z transform of a sequence of samples in discrete time as the Laplace 
transform of the continuous-time sampled function f(t) with an appropriate change of 
variable 


1 
z=e" or s=-—lnz 
T 


In Chapter 4 we saw that under this transformation the left half of the s plane, Re(s) < 0, 
is mapped onto the region inside the unit circle in the z plane, |z| < 1. This is 
consistent with our stability criteria in the s and z domains. 


Solution of discrete-time state-space equations 


The state-space approach to the analysis of continuous time dynamic systems, developed 
in Section 5.7, can be extended to the discrete-time case. The discrete form of the state- 
space representation is quite analagous to the continuous form. 


State-space model 


Consider the nth-order linear time-invariant discrete-time system modelled by the 
difference equation 


Ven + AnVen1 + An2Vien2 + + + AV, = Doug (6.67) 


which corresponds to (6.32), with b,= 0 (i > 0). Recall that {,} is the output sequence, 
with general term y,, and {u,} the input sequence, with general term u,. Following the 


procedure of Section 1.9.1, we introduce state variables x,(k), x,(k), ..., x,(k) for the 
system, defined by 
KK) = Veo 52K) = Vere +02 nC) = Vern (6.68) 


Note that we have used the notation x,k) rather than the suffix notation x,, for clarity. 
When needed, we shall adopt the same convention for the input term and write u(k) for 
u, in the interests of consistency. We now define the state vector corresponding to this 
choice of state variables as x(k) =[x,(k) x(k)... x,(k)]'. Examining the system 
of equations (6.68), we see that 


X(A + 1) = Vpn = X0(4) 


Xo(k + 1) = Vian = ¥3(K) 


XyCk + 1) = Veen = Xn(K) 
X(K + 1) = Ven 
= Gy Vern — An-2Veewn-2 — «+ — AV + Oouly 
= Ay 1X, (K) — Gy 2Xp(k) — ©. — Ag x(k) + bou(k) 


using the alternative notation for 1,. 
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Example 6.28 


Solution 


We can now write the system in the vector—matrix form 


Ix(k+1)] [0 1 0 oO... 0 JJx ce) Jol 
x(k +1) 0 0 1 OO ... 0 |ix,(| |0 
x(k+1) = =|: st : . [+]. fut’) 
; 0 0 0 0 1 : : 
x,(k +1) Ao a, a> a, ... —G,-1| |X,(k) by 
(6.69) 


which corresponds to (1.60) for a continuous-time system. Again, we can write this 
more concisely as 


x(k-+ 1) = Ax(k) + bu(k) (6.70) 


where A and b are defined as in (6.69). The output of the system is the sequence {y,}, 
and the general term y, = x,(k) can be recovered from the state vector x(k) as 


wWh=x(=f1 0 0 ... O]x(k) =e'x(k) (6.71) 


As in the continuous-time case, it may be that the output of the system is a combination 
of the state and the input sequence {u(k)}, in which case (6.71) becomes 


y(k) = e'x(k) + du(k) (6.72) 


Equations (6.70) and (6.72) constitute the state-space representation of the system, 
and we immediately note the similarity with (1.63a, b) derived for continuous-time 
systems. Likewise, for the multi-input—multi-output case the discrete-time state-space 
model corresponding to (1.69a, b) is 


x(k + 1) = Ax(’) + Bu(k) (6.73a) 

y(k) = Cx(k) + Duc’) (6.73b) 

Determine the state-space representation of the system modelled by the difference 
equation 


Viw2 + 0.2V 41 + 0.39, = Uy (6.74) 


We choose as state variables 
XK) = Ver XK) = Vet 
Thus 
x(k + 1) =x,(A) 
and from (6.74), 
x(k + 1) = —0.3x,(k) — 0.2x,(k) + u(k) 
The state-space representation is then 


x(k + 1)=Ax(k) + bu(k), (kK) = e'x(k) 
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Figure 6.19 Block 
diagram of system 
with transfer 
function G(z) = 


(z — 1)/(z* + 3z + 2). 


with 


We notice, from reference to Section 6.6.1, that the procedure used in Example 6.28 
for establishing the state-space form of the system corresponds to labelling the output 
of each delay block in the system as a state variable. In the absence of any reason for 
an alternative choice, this is the logical approach. Section 6.6.1 also gives a clue 
towards a method of obtaining the state-space representation for systems described by 
the more general form of (6.32) with m > 0. Example 6.19 illustrates such a system, 
with z transfer function 


G2)=5 z-l 
z+3z+2 


The block diagram for this system is shown in Figure 6.9(c) and reproduced for 
convenience in Figure 6.19. We choose as state variables the outputs from each delay 
block, it being immaterial whether we start from the left- or the right-hand side of the 
diagram (obviously, different representations will be obtained depending on the choice 
we make, but the different forms will yield identical information on the system). 
Choosing to start on the right-hand side (that is, with x,(4) the output of the right-hand 
delay block and x,(x) that of the left-hand block), we obtain 


x(k + 1) =x,(k) 
x(k + 1) = —3x,(k) — 2x,(k) + u(k) 
with the system output given by 
MWK) = —x)(K) + x9(k) 
Thus the state-space form corresponding to our choice of state variables is 


x(k + 1)=Ax(k)+ bulk), —-y(k) = e'x(k) 


aa| 5 Ht r= c=[-1 1] 
2 8 1 


We notice that, in contrast with the system of Example 6.28, the row vector c’ = [-1_ 1] 
now combines contributions from both state variables to form the output (A). 


with 
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Example 6.29 


Solution 


6.8 SOLUTION OF DISCRETE-TIME STATE-SPACE EQUATIONS 533 


Solution of the discrete-time state equation 


As in Section 1.10.1 for continuous-time systems, we first consider the unforced or 
homogeneous case 


x(k + 1) = Ax(k) (6.75) 
in which the input u(x) is zero for all time instants 4. Taking k = 0 in (6.75) gives 
x(1) = Ax(0) 
Likewise, taking & = 1 in (6.75) gives 
x(2) = Ax(1) = A’x(0) 


and we readily deduce that in general 
x(k) = A'x(0) (k= 0) (6.76) 


Equation (6.76) represents the solution of (6.75), and is analogous to (1.80) for the 
continuous-time case. We define the transition matrix @®(k) of the discrete-time 
system (6.75) by 

@(k) = A‘ 
and it is the unique matrix satisfying 

@(k+ 1)=A®(K), @(0)=1 
where I is the identity matrix. 


Since A is a constant matrix, the methods discussed in Section 1.7 are applicable for 
evaluating the transition matrix. From (1.34a), 


A‘ =a,(Al+ a (HA + oA? +...+0,.(DA"! (6.77) 
where, using (1.34b), the a(k) (k= 0,...,-— 1) are obtained by solving simultane- 
ously the equations 

N= Ok) + a (KA, + (MA; +...+ 0,(KAT" (6.78) 


where A, (j=1,2,...,m) are the eigenvalues of A. As in Section 1.7, if A has repeated 
eigenvalues then derivatives of A‘ with respect to A will have to be used. The method 
for determining A‘ is thus very similar to that used for evaluating e*’ in Section 1.10.3. 


Obtain the response of the second-order unforced discrete-time system 


x(k) st) 
x(k+1)= = x(k) 
x(k) = 


wie 


subject tox(0)=[1 1]. 
In this case the system matrix is 


A 29 


-1 } 
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having eigenvalues A, = } and 2, =}. Since A is a 2 x 2 matrix, it follows from (6.77) 
that 


A‘ =a,(dl+a,(OA 
with (A) and a ,(k) given from (6.78), 

Meaalh+a (kha; (j=1, 2) 
Solving the resulting two equations 

GG) = ao(k) + Gok), (GY = Gol) + (5 orl’) 
for O (k) and a,(k) gives 

Oo(k) = 365 )- 2G), (kK) = 6[G- G JT 


Thus the transition matrix is 
k 
G) 0 


D(k) = A'= k k k 
6) =) 1 


Note that ®(0) = I, as required. 
Then from (6.76) the solution of the unforced system is 


(oy 0 |] 1 ()" 
x(k + vy) = k k k - k k 
6fy- ‘TY l1) [7*- 6) 


Having determined the solution of the unforced system, it can be shown that the 
solution of the state equation (6.73a) for the forced system with input u(k), analogous 
to the solution given in (1.81) for the continuous-time system 


x=Ax+ Bu 


x(k) = A‘x(0) + Sy A‘ 'Bu(j) (6.79) 


j=0 
Having obtained the solution of the state equation, the system output or response y(k) 
is obtained from (6.73b) as 
k-1 : 
y(k) = CA'x(0) + CS AY Bu(j) + Duck) (6.80) 


j=0 


In Section 5.7.1 we saw how the Laplace transform could be used to solve the state- 
space equations in the case of continuous-time systems. In a similar manner, z trans- 
forms can be used to solve the equations for discrete-time systems. 
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6.8 SOLUTION OF DISCRETE-TIME STATE-SPACE EQUATIONS 535 
Defining Z{x(k)} = X(z) and Z{u(k)} = U(z) and taking z transforms throughout in 
the equation 
x(k + 1) = Ax(k) + Bu(k) 
gives 
zX(z) — zx(0) = AX(z) + BU(z) 
which, on rearranging, gives 
(zl — A)X(z) = zx(0) + BU(z) 
where I is the identity matrix. Premultiplying by (zl— A)" gives 
X(z) = 2(zl — A)'x(0) + (zl — A) 'BU(z) (6.81) 


Taking inverse z transforms gives the response as 
x(k) = £'X(z)} = Fl z(2l — AY'}x(0) + FM (zl - A) 'BUG)} (6.82) 


which corresponds to (5.89) in the continuous-time case. 
On comparing the solution (6.82) with that given in (6.79), we see that the transition 
matrix @(f) = A‘ may also be written in the form 


@(t) = A‘ = ¥1{2(zl- A)'} 


This is readily confirmed from (6.81), since on expanding z(zl — A)" by the binomial 
theorem, we have 


2 ig 
ad-Ay'=144484...4A44 
Zz Zz Zz 
=y 4-204} 


Using the z-transform approach, obtain an expression for the state x(k) of the system 
characterized by the state equation 


x(k +1)= aes x(k) + : u(k) (k= 0) 
-3 -6 1 
when the input is the unit step function 


ut) (k < 0) 
1 (k=0) 


and subject to the initial condition x(0)=[1 —1]’. 
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Solution In this case 


giving 
(:l- Ay! = 1 z+6 5 
(ee Ie43)) 3. ga 9 
5 3 5 5 
2 2. _2__ _2_ 
_|zZ+1 z+3 zt+1 2+3 
3 3 a2 3 
204 2 24 2 
z+1 24+3 z4+1 2+4+3 
Then 


F {22h - AY'} =F"! 


3(-1)'- 3-3)" 8-1)" - 3(-3)5 
-3(-1)' + $(-3)'  -3(-1)" + 3(-3)* 


so that, with x(0)=[1 —1]', the first term in the solution (6.82) becomes 


F{2(l — AY}x(0) = eae 
—(-3) 


Since U(z) = F{u(k)} =2z/(z - 1), 


ee ee ea | oe 
(zl — A) Boe Ga iara| 3 ba Hes 


_ Zz z+1l 
~ (z-1)(z + 1I(z + 3)[z—5 


iw 


z+1 24+3 
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so that the second term in the solution (6.82) becomes 


pole ay 
#"s(2l — A) 'BU(z)} = ; : ; (6.84) 
7G =G3y 


Combining (6.83) and (6.84), the response x(k) is given by 


3 _ 8(-1)* + 2(-3)' 


2 


x(k) = 


$+ 3(-1)" - 2(-3)' 


Having obtained an expression for a system’s state x(f), its output, or response, y(*) may 
be obtained from the linear transformation (6.73b). 


6.8.3 Exercises 
Check your answers using MATLAB or MAPLE whenever possible. 


Use z transforms to determine A‘ for the matrices when the input is the unit step function 
0 1 -1 3 -1 1 0 (k<0 

(a) (b) (©) u(k) -| sane 
4 0 3 -l 0 -l 1 (k= 0) 


. ; : ject to the initial iti =[1l -l]’. 
Solve the discrete-time system specified by end cubjech to Mie aieeh Cendinon = | 


x(k + 1) = —Tx(k) + Ay(k) 33 The difference equation 


ANS TYG) TIRE vk+2)=yk+ +9) 
with x(0) = 1 and y(0) = 2, by writing it in the form 
x(k + 1) = Ax(k). Use your answer to calculate x(1) 
and x(2), and check your answers by calculating 
x(1), v(1), x(2), y(2) directly from the given 
difference equations. 


with 1(0) = 0, and )(1) = 1, generates the Fibonacci 
sequence { y(k)}, which occurs in many practical 
situations. Taking x(k) = (A) and x,(k) = v(k + 1), 
express the difference equation in state-space form 
and hence obtain a general expression for y(k). 
Show that as k — the ratio y(k + 1)/y(k) tends 
to the constant } (5 + 1). This is the so-called 
Golden Ratio, which has intrigued mathematicians 
for centuries because of its strong influence on art 
0 i 1 and architecture. The Golden Rectangle, that is one 
x(k+1)= x(k) + | u(k) whose two sides are in this ratio, is one of the most 
0.16 — 1 visually satisfying of all geometric forms. 


Using the z-transform approach, obtain an 
expression for the state x(k) of the system 
characterized by the state equation 
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6.9.1 


Discretization of continuous-time state-space models 


In Sections 1.10 and 5.7 we considered the solutions of the continuous-time state-space 
model 


X(t) = Ax(t) + Bu(t) (6.85a) 
y(t) = Cx(f) (6.85b) 


If we wish to compute the state x(f) digitally then we must first approximate the continuous 
model by a discrete-time state-space model of the form 


x[(k + 1)T] = Gx(kT) + Hu(kT) (6.86a) 
WKT) = Cx(kT) (6.86b) 
Thus we are interested in determining matrices G and H such that the responses to the 
discrete-time model (6.86) provide a good approximation to sampled-values of the 
continuous-time model (6.85). We assume that sampling occurs at equally spaced 


sampling instances ¢t = kT, where T > 0 is the sampling interval. For clarification we 
use the notation x(AT) and x[(k + 1)7] instead of & and (K+ 1) as in (6.73). 


Euler’s method 


A simple but crude method of determining G and H is based on Euler’s method con- 
sidered in Section 10.6 of Modern Engineering Mathematics. Here the derivative of the 
state is approximated by 


x(t +7) -x(T 


x(t) = T 


which on substituting in (6.85a) gives 


x(t +7) - x(t 


T = Ax(t) + Bu(t) 

which reduces to 

x(t + T)=(TA + I)x(t) + TBu(t) (6.87) 
Since ¢ is divided into equally spaced sampling intervals of duration T we take t = kT, 
where & is the integer index k= 0, 1, 2,..., so that (6.87) becomes 

x[(k + 1)T] = (TA + D)x(kT) + TBu(kT) (6.88) 
Defining 

G=G,=(TA+J/) and H=H,=T7B (6.89) 


(6.86) then becomes the approximating discrete-time model to the continuous-time 
model (6.85). This approach to discretization is known as Euler’s method and simply 
involves a sequential series of calculations. 
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Example 6.31 Consider the system modelled by the second-order differential equation 
V(t) + 39D) + 2y = 2u(t) 
(a) Choosing the state-vector x = [y y]" express this in a state-space form. 


(b) Using Euler’s method, determine the approximating discrete-time state-space 
model. 


(c) Illustrate by plotting the responses y(t), for both the exact continuous response 
and the discretized responses, for a step input u(f) = 1 and zero initial conditions, 
taking T= 0.2 


Solution (a) Since x, = y, x =) we have that 
X= Y=Xy 
Xo = P=—2x, — 3x, + 2u 


so the state-space model is 
AYO WS 
Xy -2  -3)|*2 2 

x 
y= [1 af | 

X2 


(b) From (6.89) 


@eTdxrs|:! r 
oT eae 


=| 
oT 


so the discretized state-space model is 


x [AFIT] _ | 1 T | \x(AT) 7 
x2[(k +1)T] -2T -37+1||x(AT) 
WET) = [1 fein 


2 u(kT) 


x,(kT) 


2) (c) Using the MATLAB commands: 


A = [0,1;-2,-3]3; B = [0;2]; C = [1,0]; 


Gl = [1,T}H2*T, =3*7T+1)7 Hl = 10;2*T] 3 
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Figure 6.20 
Discretization using 
Euler’s method. 


6.9.2 


T = T*[0:30]; 

y = step(A,B,C,0,1,t); yd = dstep(G1,H1,C,0,1,31); 
plot(t,y,t,yd,'’x’) 

end 


step responses for both the continuous model and the Euler discretized model are 
displayed in Figure 6.20 with ‘x’ denoting the discretized response. 


Step response 


Step-invariant method 


To determine the matrices G and H in the discrete-time model (6.86), use is made of 
the explicit solution to the state equation (6.85a). From (1.81) the solution of (6.85a) is 
given by 


A(t-T)) 


x(t) = 6 x(t) + | eo Bu(t,) dt, (6.90) 


% 
Taking ft) = AT and t= (k + 1)T in (6.90) gives 


(k+1)T 
Al(k+1)T-1)] 


x[(k+1)T] = ein +| Bu(t,) dt, 


kT 
Making the substitution t= tT, — kT in the integral gives 


T 


x[(k+1)T] = ein +| eT Bu(kT+ 7) dt (6.91) 


0 
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The problem now is: How do we approximate the integral in (6.91)? The simplest 
approach is to assume that all components of u(t) are constant over intervals between 
two consecutive sampling instances so 


u(kT + 1) =u(kT), O<ST<T, k=0,1,2,... 


The integral in (6.91) then becomes 


| eR a u(kT) 


Defining 
G=e"" (6.92a) 
T T 
and H= | eB dt = | e'B dt, using substitution t=(T-— 7) (6.92b) 
0 0 
then (6.91) becomes the discretized state equation 
x[(k + 1)T] = Gx(kT) + Hu(kT) (6.93) 


The discretized form (6.93) is frequently referred to as the step-invariant method. 


Comments 
1. From Section (5.7.1) we can determine G using the result 
ec = £'{(sl- AY} (6.94) 


2. If the state matrix A is invertible then from (1.37) 
T 
H= | eB dt = A\(G-)B=(G-NA'B (6.95) 
0 


3. Using the power series expansion of e“ given in (1.27) we can express G and H 
as the power series 


TR _ 0 T", r 
G=14+7A+ mI t= > (6.96) 
r=0 
2 eo Tr par-l 
H=(14 ra ee [pT (6.97) 
. r=1 : 


We can approximate G and H by neglecting higher-order terms in 7. In the par- 
ticular case when we neglect terms of order two or higher in T results (6.97) give 


G=1+ TAand H= 7B 


which corresponds to Euler’s discretization. 
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Example 6.32 


Solution 


Using the step-invariant method, obtain the discretized form of the state equation for 
the continuous-time system 


x= ie — 0 1) | + 0 u(t) 
X -2 -3)|*X2 2 
considered in Example 6.31. Plot the response (kT) = [1 O]x(AT), for a step input 


u(t) = | and zero initial conditions, taking T= 0.2. 


T 


Using (6.93) G = e*” and H = | eB dt. From (6.94) 


0 


G= olay) =se{ 1)? i A= (42)841) 


A) =2. 5 

1 2, 1 nn 1 
=| st+2 stl s+2 stl 

2 2 2 1 


so that 


and 


Thus, the discrete form of the state equation is 


-2T =. 


-2e° +e 


-e° +2e se 46 | 
Oe = 96° 26 He 


sero] ei xan “eo 


In the particular case 7 = 0.2 the state equation is 


-0.2968 0.5219 


x{(k-+ 10.2] = | 0.9671 nana?) + | Gt 


Jao 


Using MATLAB step responses for both the continuous-time model and the discretized 
step-invariant model are displayed in Figure 6.21, with ‘x’ denoting the discretized 
response. 
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step-invariant method. 
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Step response 


For a given value of 7 the matrices G and H may be determined by the step-invariant 
method using the MATLAB function c2d (continuous to discrete). Thus, for the 
system of Example 6.32 with T= 0.2, the commands 


= LO, g=2_—3 lf 
Bos [Ue 2] 
[G,H] = c2d(A,B,0.2) 
return 
G@= 0.9571 0.1484 
=O 2958 0.5219) 
if = 0.0329 
(0), 2 Mats 


which checks with the answers given in Example 6.32. 


6.9.3 Exercises 


Using the step-invariant method obtain the discretized 
form of the continuous-time state-equation 


EJ-B ae fe 


Check your answer using MATLAB for the 
particular case when the sampling period is T= 1. 


y=[1 O]x 


(a) Determine the Euler form of the discretized 
state-space model. 


(b) Determine the discretized state-space model 


An LCR circuit, with L = C= R = 1, may be 
modelled by the continuous-time state-space model 
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(c) Using MATLAB plot, for each of the three 
models, responses to a unit step input u(t) = 1 
with zero initial conditions, taking the 
sampling period T= 0.1. 


36 A linear continuous-time system is characterized by 
the state matrix 


Ld 


(a) Show that the system is stable. 


(b) Show that the state matrix of the corresponding 
Euler discrete-time system is 


Pee 1-T fi 
-fF 1-27 
(c) Show that stability of the discretized system 


requires T < 1. 


37 A simple continuous-time model of a production 
and inventory control system may be represented by 
the state-space model 


ep _ |¥1(2) 

X(t) = Be 

-1 Of /x1@ ah 0} | (2) 
1 0} |x2(t) 0 -1}}us(t) 


where x,(f) represents the actual production rate and 
x,(t) represents the current inventory level; w,(4) 


represents the scheduled production rate, u,(f) 
represents the sales rate and k, is a constant 
gain factor. 


(a) Determine, using the step-invariant method, 
the discretized form of the model. Express the 
model in the particular case when the sampling 
period T= 1. 


(b) Suppose the production schedule is determined 
by the feedback policy 


u(kT) =k, — x,(kT) 


where k, is the desired inventory level. The 
system is originally in equilibrium with x,(0) 
equal to the sales rate and x,(0) = k,. At time 
t= 0 the sales rate suddenly increases by 
10%; that is, u,(¢) = 1.1x,(0) for t= 0. Find 
the resulting discrete-time state model, with 
sampling rate 7 = 1 and taking k, = 2. 


(c) Find the response of the given continuous-time 
model, subject to the same feedback control 


policy 
u,(t) = k, — x,(t) 
and the same initial conditions. 


The exercise may be extended to include simulation 
studies using MATLAB. 


(This exercise is adapted from an illustrative 
problem in William L. Brogan, Modern Control 
Theory, 2™ edition, Prentice-Hall, 1985.) 


mtn ardinclalicalyeieciiem cesign of discrete-time 


systems 


An important development in many areas of modern engineering is the replacement 
of analogue devices by digital ones. Perhaps the most widely known example is the 
compact disc player, in which mechanical transcription followed by analogue signal 
processing has been superseded by optical technology and digital signal processing. 
Also, as stated in the introduction, DVD players and digital radios are setting new 
standards in home entertainment. There are other examples in many fields of engineering, 
particularly where automatic control is employed. 
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Figure 6.22 
Amplitude response 
for an ideal low-pass 
filter. 


Figure 6.23 

LCR network for 
implementing a 
second-order 
Butterworth filter. 
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Analogue filters 


At the centre of most signal processing applications are filters. These have the effect 
of changing the spectrum of input signals; that is, attenuating components of signals 
by an amount depending on the frequency of the component. For example, an analogue 
ideal low-pass filter passes without attenuation all signal components at frequencies 
less than a critical frequency @ = @, say. The amplitude of the frequency response 
|G(jq@) | (see Section 5.8) of such an ideal filter is shown in Figure 6.22. 

One class of analogue filters whose frequency response approximates that of the 
ideal low-pass filter comprises those known as Butterworth filters. As well as having 
‘good’ characteristics, these can be implemented using a network as illustrated in 
Figure 6.23 for the second-order filter. 

It can be shown (see M. J. Chapman, D. P. Goodall and N. C. Steele, Signal Processing 
in Electronic Communication, Horwood Publishing, Chichester, 1997) that the transfer 
function G,(s) of the nth-order filter is 


n 


1 k 
G,(s) =——~ where 8B,(x) = a,x 
B,(x) 2 : 
with 
Ss * cos(r- 1a Tt 
x= —, em ; > a=— 
QO, 7m I] sin ra 2n 


Using these relations, it is readily shown that 


2 
@ 
G(s) = ee (6.98) 
S +\20.8+ Of 

3 
G(s) = Le (6.99) 


3 2 2. 
s+ 20,5 +2@25+ 


and so on. On sketching the amplitudes of the frequency responses G,,(j@), it becomes 
apparent that increasing n improves the approximation to the response of the ideal 
low-pass filter of Figure 6.22. 


IGQGuw)| 
1 
We, oO We w 
R L 
t a 
u(t) Cy C2 —— y(t) 
v v 
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6.10.2 


Designing a digital replacement filter 


Suppose that we now wish to design a discrete-time system, to operate on samples 
taken from an input signal, that will operate in a similar manner to a Butterworth filter. 
We shall assume that the input signal u(¢) and the output signal y(f) of the analogue filter 
are both sampled at the same intervals 7 to generate the input sequence {u(kT)} and 
the output sequence {)(AT)} respectively. Clearly, we need to specify what is meant 
by ‘operate in a similar manner’. In this case, we shall select as our design strategy a 
method that matches the impulse response sequence of the digital design with a 
sequence of samples, drawn at the appropriate instants T from the impulse response of 
an analogue ‘prototype’. We shall select the prototype from one of the Butterworth 
filters discussed in Section 6.10.1, although there are many other possibilities. 

Let us select the first-order filter, with cut-off frequency @,, as our prototype. Then 
the first step is to calculate the impulse response of this filter. The Laplace transfer 
function of the filter is 


QO. 


G — 
st+Q, 


So, from (5.71), the impulse response is readily obtained as 
A(t)=@,e°° (t=0) (6.100) 
Next, we sample this response at intervals T to generate the sequence 
{H(KT)} = {ae} 


which on taking the z transform, gives 


F{h(kT)} = H(z) = @.— 


-O T 

Finally, we choose H(z) to be the transfer function of our digital system. This means 

simply that the input-output relationship for the design of the digital system will be 
Y(z) = H(z)U(z) 


where Y(z) and U(z) are the z transforms of the output and input sequences {y(kT)} 
and {u(kT)} respectively. Thus we have 


Y(z) = @, one U2) (6.101) 


Z-e¢ 


Our digital system is now defined, and we can easily construct the corresponding 
difference equation model of the system as 


(g-e)¥(z) = w,zU(2) 
that is 
zY(z) - e” ¥(z) = @,zU(z) 


Under the assumption of zero initial conditions, we can take inverse transforms to obtain 
the first-order difference equation model 


y(k+1)—e yk) = @ulk + 1) (6.102) 
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Figure 6.24 Block 
diagram for the digital 
replacement filter, 
a=ko,, B=". 


A block diagram implementation of (6.102) is shown in Figure 6.24. 


yk) 


u(k) © + é& eg) 


+ 


6.10.3 Possible developments 


6.11 


6.11.1 


The design method we have considered is called the impulse invariant technique, 
and is only one of many available. The interested reader may develop this study in 
various ways: 


(1) 


(2) 
(3) 


(4) 


(5) 


Write a computer program to evaluate the sequence generated by (6.102) with 
@, = 1, and compare with values obtained at the sampling instants for the impulse 
response (6.100) of the prototype analogue filter. 


Repeat the cond-order Butterworth filter. 

By setting nsfer function of the prototype, and z = e!®” 
in the z tr gital design, compare the amplitude of the 
frequency For an explanation of the results obtained, 
see Chapte 

An alterna replace s in the Laplace transfer function 
with 

2z-1 

Tz+1 


(this is a process that makes use of the trapezoidal method of approximate 
integration). Design alternative digital filters using this technique, which is 
commonly referred to as the Tustin (or bilinear transform) method (see 
Section 6.11.3). 


Show that filters designed using either of these techniques will be stable provided 
that the prototype design is itself stable. 


aediaccguccarl yeeros the delta operator and 


the & transform 


Introduction 


In recent years, sampling rates for digital systems have increased many-fold, and tradi- 
tional model formulations based on the z transform have produced unsatisfactory 
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6.11.2 


results in some applications. It is beyond the scope of this text to describe this situation 
in detail, but it is possible to give a brief introduction to the problem and to suggest an 
approach to the solution. For further details see R. M. Middleton and G. C. Goodwin, 
Digital Control and Estimation, A Unified Approach (Prentice Hall, Englewood Cliffs, 
NJ, 1990) or W. Forsythe and R. M. Goodall, Digital Control (Macmillan, London, 1991). 
The contribution of Colin Paterson to the development of this application is gratefully 
acknowledged. 


The q or shift operator and the 6 operator 


In the time domain we define the shift operator q in terms of its effect on a sequence 
{x,} as 


qixit = (Xas} 


That is, the effect of the shift operator is to shift the sequence by one position, so that 
the Ath term of the new sequence is the (k + 1)th term of the original sequence. It is then 
possible to write the difference equation 


Vew2 + Wear + Ve = Win — Ue 
as 
Gy, + 2g), + 5; = qu, — Uy 
or 
(q+ 2q+5)y=(q- Dm (6.103) 


Note that if we had taken the z transform of the difference equation, with an initially 
quiescent system, we would have obtained 


(2? + 22+ 5)¥(z) =(z- 1)U(2) 


We see at once the correspondence between the time-domain q operator and the 
z-transform operator &. 
The next step is to introduce the 6 operator, defined as 


where A has the dimensions of time and is often chosen as the sampling period T. Note 
that 


= ys Vea Ye 
Su. = (q k _ Vier 7 Vs 
Vk i ik 


so that if A = T then, in the limit of rapid sampling, 


~ dy 
oy dt 


Solving for q we see that 


q=1+A6 
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6.11.3 


The difference equation (6.103) can thus be written as 
((1 + Ady + 2(1 + Ad) + 5)y, = [C1 + AS) — 1], 
or 
[(Ad)’ + 4A6 + 8] y, = Adu, 


or, finally, as 


Constructing a discrete-time system model 


So far, we have simply demonstrated a method of rewriting a difference equation in an 
alternative form. We now examine the possible advantages of constructing discrete- 
time system models using the 6 operator. To do this, we consider a particular example, 
in which we obtain two different discrete-time forms of the second-order Butterworth 
filter, both based on the bilinear transform method, sometimes known as Tustin’s 
method. This method has its origins in the trapezoidal approximation to the integra- 
tion process; full details are given in M. J. Chapman, D. P. Goodall and N. C. Steele, 
Signal Processing in Electronic Communication (Horwood Publishing, Chichester, 
1997). 

The continuous-time second-order Butterworth filter with cut-off frequency @, = | 
is modelled, as indicated by (6.98), by the differential equation 


2 
dy 41414 ni aya uty (6.104) 
dt dt 


where u(t) is the input and y(?) the filter response. Taking Laplace transforms through- 
out on the assumption of quiescent initial conditions, that is y(0) = (dy/df)(0) = 0, we 
obtain the transformed equation 


(s°+ 1.414 21s + 1)¥(s) = Us) (6.105) 
This represents a stable system, since the system poles, given by 
s°+141421s+1=0 


are located at s = —0.707 10 + j0.707 10 and thus lie in the left half-plane of the complex 
s plane. 

We now seek a discrete-time version of the differential equation (6.104). To do this, 
we first transform (6.105) into the z domain using the bilinear transform method, 
which involves replacing s by 


2z-1 
Tz+1 


Equation (3.74) then becomes 


4 & 1) + 1.414 212 (2 t)+ Y¥(z) = U(z) 


+1 Zz 
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or 
(G7? + 141421 x $7+4274+ (G7? - 824+ 47’ - 1.41421 x $7+ 4] %2) 
=17'(¢ + 227+ 1)U@) (6.106) 
We can now invert this transformed equation to obtain the time-domain model 
(47? + 1.41421 x $ T+ 4)yuo + GT? - 8)yin + GT? — 1.41421 x 5 T+ 4)y, 
= 17? (yyy + Qty + Hy) (6.107) 
For illustrative purposes we set 7 = 0.1 s in (6.107) to obtain 
4.07321 y,45 — 7.995 00y,,, + 3.931 79y, = 0.025 00(uj45 + 2.) + U,) 


Note that the roots of the characteristic equation have modulus of about 0.9825, and are 
thus quite close to the stability boundary. 
When 7 = 0.01 s, (6.107) becomes 


4.007 10) 49 — 7.999 95,41 + 3.992 95; = 0.000 03(tdg49 + 2etps1 + Uy) 


In this case the roots have modulus of about 0.9982, and we see that increasing the 
sampling rate has moved them even closer to the stability boundary, and that high 
accuracy in the coefficients is essential, thus adding to the expense of implementation. 

An alternative method of proceeding is to avoid the intermediate stage of obtaining 
the z-domain model (6.106) and to proceed directly to a discrete-time representation 
from (6.104), using the transformation 


_,2q-1 
Tq+l 


leading to the same result as in (6.107). Using the 6 operator instead of the shift operator 
q, noting that q = 1 + Ad, we make the transformation 


2_A6 
T2+A6 


or, if T= A, the transformation 


26 
2+A6 


So? 


in (6.105), which becomes 
[6° + 1.41421 x +6(2 + Ad) + i(2 + Ad)]),= 4(2 + Ad)u, 


Note that in this form it is easy to see that in the limit as A > 0 (that is, as sampling 
becomes very fast) we regain the original differential equation model. Rearranging this 
equation, we have 


34 (1.414 21+A) 54 1 
k 
(1+1.41421xiA+4A*) (14+ 1.41421 x $A + 4A’) 
2 
+ A6N (6.108) 


i u 
4(1+1.41421xtA+ia’) © 
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6.11.4 


Figure 6.25 
The 6" block. 


In order to assess stability, it is helpful to introduce a transform variable y associated 
with the 6 operator. This is achieved by defining yin terms of z as 


oe taal 


eh 


The region of stability in the z plane, |z| < 1, thus becomes 
|l+Ay| <1 


or 
i + 7| ai (6.109) 
A A 


This corresponds to a circle in the y domain, centre (—1/A, 0) and radius 1/A. As 
A — 0, we see that this circle expands in such a way that the stability region is the 
entire open left half-plane, and coincides with the stability region for continuous-time 
systems. 

Let us examine the pole locations for the two cases previously considered, namely 
T=0.1 and T= 0.01. With A = T= 0.1, the characteristic equation has the form 


¥ + 1.41092y + 0.93178 = 0 


with roots, corresponding to poles of the system, at —0.705 46 + 0.658 87. The centre 
of the circular stability region is now at —1/0.1 = —10, with radius 10, and these roots 
lie at a radial distance of about 9.3178 from this centre. Note that the distance of 
the poles from the stability boundary is just less than 0.7. The poles of the original 
continuous-time model were also at about this distance from the appropriate boundary, 
and we observe the sharp contrast from our first discretized model, when the discretiza- 
tion process itself moved the pole locations very close to the stability boundary. In 
that approach the situation became exacerbated when the sampling rate was increased, 
to T= 0.01, and the poles moved nearer to the boundary. Setting 7 = 0.01 in the new 
formulation, we find that the characteristic equation becomes 


Y + 1.414 13y + 0.99295 =0 


with roots at —0.707 06 + j0.702 14. The stability circle is now centred at —100, with 
radius 100, and the radial distance of the poles is about 99.2954. Thus the distance from 
the boundary remains at about 0.7. Clearly, in the limit as A > 0, the pole locations 
become those of the continuous-time model, with the stability circle enlarging to 
become the entire left half of the complex y plane. 


Implementing the design 


The discussion so far serves to demonstrate the utility of the 6 operator formulation, but 
the problem of implementation of the design remains. It is possible to construct a 6"! 
block based on delay or 1/z blocks, as shown in Figure 6.25. Systems can be realized 
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using these structures in cascade or otherwise, and simulation studies have produced 
successful results. An alternative approach is to make use of the state-space form of 
the system model (see Section 6.18). We demonstrate this approach again for the case 
T=0.01, when, with T= A = 0.01, (6.108) becomes 


(5° + 1.414 136+ 0.992 95)y, 

= (0.000 028 + 0.009 306 + 0.992 95)u, (6.110a) 
Based on (6.110a) we are led to consider the equation 

(& + 1.414 136+ 0.992 95)p, = u, (6.110b) 
Defining the state variables 

Mik = Pro Xo4 = ODy 
equation (6.110b) can be represented by the pair of equations 

OX), = Xp 

6X24 = —0.992 95x, ,— 1.414 13x, + Uy 
Choosing 

VY, = 0.992 95p, + 0.009 30dp, + 0.000 0025°p, (6.110c) 


equations (6.110b) and (6.110c) are equivalent to (6.110a). In terms of the state 
variables we see that 


, = 0.992 93x; + 0.009 72x, + 0.000 021, 


Defining the vectors x, = [x;,  2,J' and 6x, =[6x,,  &x,,J", equation (6.11 1a) can be 
represented in matrix form as 


OX, = 0 ; x,t ux (6.111a) 
-0.99295 -1.41413 1 
with 
y, = [0.99293 0.009 72}x; + 0.000 020, (6.111b) 


We now return to the q form to implement the system. Recalling that 6 = (q — 1)/A, 


(6.111a) becomes 
GX, = Xeni = XEK A ; x, + a uy (6.112) 
-0.99295 -1.41413 1 


with (6.111b) remaining the same and where A = 0.01, in this case. Equations (6.112) 
and (6.111b) may be expressed in the vector—matrix form 


Xiu1 =X; + ATA(A)x, + bu] 


Vs c'(A)x, + d(A)u, 
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6.11.5 


This matrix difference equation can now be implemented without difficulty using 
standard delay blocks, and has a form similar to the result of applying a simple Euler 
discretization of the original continuous-time model expressed in state-space form. 


The & transform 


In Section 6.11.3 we introduced a transform variable 
ooo 
A 


The purpose of this was to enable us to analyse the stability of systems described in the 
6 form. We now define a transform in terms of the z transform using the notation given 
by R. M. Middleton and G. C. Goodwin, Digital Control and Estimation, A Unified 
Approach (Prentice Hall, Englewood Cliffs, NJ, 1990). Let the sequence { f,} have z 
transform F(z); then the new transform is given by 


FAY) = F@) leaps 
e z 


The & transform is formally defined as a slight modification to this form, as 


Df) = FY) = AFA) 


= fh 
=AYyY—~* 
ay 


k=0 


The purpose of this modification is to permit the construction of a unified theory of 
transforms encompassing both continuous- and discrete-time models in the same 
structure. These developments are beyond the scope of the text, but may be pursued 
by the interested reader in the reference given above. We conclude the discussion 
with an example to illustrate the ideas. The ramp sequence {u,} = {kA} can be 
obtained by sampling the continuous-time function /(‘) = ¢ at intervals A. This sequence 
has z transform 


U(z) = 


(z-1)° 


and the corresponding & transform is then 


AUK(y) = 4Y 
id 


Note that on setting A = 0 and y = s one recovers the Laplace transform of f(f). 
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6.11.6 Exercises 


38 A continuous-time system having input »(t) and 
output y(t) is defined by its transfer function 


1 
(s+ 1)(s +2) 


Use the methods described above to find the q and 
6 form of the discrete-time system model obtained 
using the transformation 


H(s) = 


2z-1 


Az+1 


where A is the sampling interval. Examine the 
stability of the original system and that of the 
discrete-time systems when A = 0.1 and when 
A=0.01. 


39 Use the formula in equation (6.99) to obtain the 
transfer function of the third-order Butterworth 
filter with @, = 1, and obtain the corresponding 
6 form discrete-time system when T= A. 


40 = Make the substitution 


xD =O 
_ dy) 
X,(t) = dt 


in Exercise 38 to obtain the state-space form of the 
system model, 


X(t) = Ax(t) + bu(t) 


41 


y(t) = e'x(t) + du(t) 


The Euler discretization technique replaces x(t) by 


x((k + 1)A) - x(kA) 
A 


Show that this corresponds to the model obtained 
above with A = A(0), c = c(0) and d=d(0). 


The discretization procedure used in Section 6.11.3 
has been based on the bilinear transform method, 
derived from the trapezoidal approximation to the 
integration process. An alternative approximation 
is the Adams—Bashforth procedure, and it can be 
shown that this means that we should make the 


transformation 
12. 2-z 

Se oe a 

A 5z+8z-1 


where A is the sampling interval (see W. Forsythe 
and R. M. Goodall, Digital Control, Macmillan, 
London, 1991). Use this transformation to 
discretize the system given by 


Ha 
) stl 


when A = 0.1 in 


(a) the z form, and 
(b) the yform. 


6.12 Review exercises (1-18) 


Check your answers using MATLAB or MAPLE whenever possible. 


1 The signal f(t) = ¢ is sampled at intervals T to 
generate the sequence { /(kT)}. Show that 


Bi f(kT} = ES 
(e- 


iby: 
2 Show that 
fa" sin ko} = 2S __- (q > 0) 
z -2azcos@+a 
3 Show that 
oe} = 22+) 


(z- 1) 


4 


Find the impulse response for the system with 
transfer function 


2 
je 24 


JFK) = 5 
z—-2z+1 


Calculate the step response for the system with 
transfer function 


1 


LEC) 
z+3z4+2 


A process with Laplace transfer function 
H(s) = 1/(s + 1) is in cascade with a zero-order 
hold device with Laplace transfer function 
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G(s) = (1 — e*")/s. The overall transfer function 
is then 
1-e" 
(sar ID) 
Write F(s) = 1/s(s + 1), and find f() = £7'{F(s)}. 
Sample /(#) at intervals T to produce the 
sequence { f(kT)} and find F(z) = F{ f(kT)}. 
Deduce that 


e"F(s) > LF@) 


and hence show that the overall z transfer function 
for the process and zero-order hold is 


A system has Laplace transfer function 


svar Il 


ONS (ea 


Calculate the impulse response, and obtain the 
z transform of this response when sampled at 
intervals T. 


It can be established that if X(z) is the z transform 
of the sequence {x,} then the general term of that 
sequence is given by 


j2n 
Cc 


a= if X(z)2"" dz 


where C is any closed contour containing all 

the singularities of X(z). If we assume that all the 
singularities of X(z) are poles located within a circle 
of finite radius then it is an easy application of the 
residue theorem to show that 


x, = & [residues of X(z)z”"' at poles of X(z)] 


(a) Let X(z) = 2z/(z — a)(z — b), with a and b real. 
Where are the poles of X(z)? Calculate the 
residues of z” \X(z), and hence invert the 
transform to obtain {x,,}. 

(b) Use the residue method to find 


: 1 4 aS 1 4 
es | cee | 


The impulse response of a certain discrete-time 
system is {(—1)* — 2"}. What is the step response? 
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A discrete-time system has transfer function 


2 


eee. ee! 
BO Cena 


Find the response to the sequence {1,—-1, 0,0,...}. 


Show that the response of the second-order 
system with transfer function 


2) 
Zz 


(z- a)(z - B) 

to the input (1, -(@+ B), a@B, 0, 0, 0,...} is 
Kot = lO; Ona 

Deduce that the response of the system 
eee Aaa, 
(z- a)(z - B) 

to the same input will be 
Koha = 40), 1,0, ©, coef 


A system is specified by its Laplace transfer 
function 


ou —__ 
(Gosh ae 2) 
Calculate the impulse response y(t) = £'{H(s)}, 
and show that if this response is sampled at 
intervals T to generate the sequence {ys(nT)} 
(S02 Se) then 


Ms 


PK ih 
Za Zac 


D(z) = Z{ ya(nT )} = 


A discrete-time system is now constructed so that 


¥(z) = TD@)X(Z) 


where X(z) is the z transform of the input 
sequence {x,} and Y(z) that of the output 
sequence {y,}, with x, =x(nT) and y, = (nT). 
Show that if T= 0.5 s then the difference 
equation governing the system is 


Vn — 0.9744y,,, + 0.2231, 
= 0.5%,49— 0.4226x,41 


Sketch a block diagram for the discrete-time 
system modelled by the difference equation 


Prv2— 0.9744) ,.41 + 0.223 Ip, =X), 


and verify that the signal y,, as defined above, is 
generated by taking y, = 0.5p,,,. — 0.4226p,,, as 
output. 
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13 


14 


In a discrete-time position-control system the 

position y, satisfies the difference equation 
Vier =Y, + av, (a constant) 

where v, and w, satisfy the difference equations 
V4) =v, + bu, (6 constant) 

u, = k(x, — y,) — kv, 


(a) Show that if k, = 1/4ab and k, = 1/b then the 
z transfer function of the system is 
ey 2 
X(z) (1-22) 


where Y(z) = F{y,} and X(z) = F{x,}. 


(k,, k, constants) 


(b) If also x, =A (where A is a constant), 
determine the response sequence {y,} given 
that y) =y, =0. 


The step response of a continuous-time system is 
modelled by the differential equation 


dy 3B 42y=1 (t = 0) 
dr dt 


with y(0) = (0) = 0. Use the backward-difference 
approximation 


dy _ Ve7 Vis 

dt T 

d’y we VET Ve + Vien 

dr ie 
to show that this differential equation may be 
approximated by 


Vie ~ 2Ve1 AV e-n a, 32k ill +2y,=1 
i 


Tia 
Take the z transform of this difference equation, 
and show that the system poles are at 


1 
C= 
Wap ae 


1 
zZ=—, 
hates 


Deduce that the general solution is thus 


1 k 1 k 
y= a(t) Gea uaa 


Show that y = 5 and, noting that the initial 
conditions (0) = 0 and (0) = 0 imply 
Vo = y_, = 0, deduce that 


f 2 A fie en 2 


Nie 


Vie= 


tS 


16 


Note that the z-transform method could be used to 
obtain this result if we redefine #{ y,} = Yj2-1(y,/z’), 
with appropriate modifications to the formulae for 
FAV ie} and ZlYp2}- 

Explain why the calculation procedure is 
always stable in theory, but note the pole 
locations for very small T. 

Finally, verify that the solution of the 
differential equation is 


yO) = § (e* — 2e* + 1) 


and plot graphs of the exact and approximate 
solutions with T= 0.1 s and T=0.05 s. 


Again consider the step response of the system 
modelled by the differential equation 
2 
eee eal SU) 
dr’ dt 
with (0) = 3(0) = 0. Now discretize using the 


bilinear transform method; that is, take the 
Laplace transform and make the transformation 


2 = Il 
Tz+1 


where 7 is the sampling interval. Show that the 
poles of the resulting z transfer function are at 


1-T 
zZ=—_, 
1+T7 


Det 
zZ= 
Dae de 
Deduce that the general solution is then 
LEIP | OLE 
= — | + B\/—— |] + 
_ aly) BG ¥ 


Deduce that y = ! and, using the conditions 
Yo =y_, = 0, show that 


Plot graphs to illustrate the exact solution and 
the approximate solution when T= 0.1 s and 
T=0.05 s. 


Show that the z transform of the sampled version 
of the signal f(f) = 7? is 


F(z) = 22+ DA" 
(z- 1) 


where A is the sampling interval. Verify that 
the & transform is then 
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18 


(1. +Av)(2+ Av) 


3 
v 


Show that the eigenvalues of the matrix 


1 —2 
A=/-1 2 1 
@ il sll 


are 2, 1 and —1, and find the corresponding 

eigenvectors. Write down the modal matrix Mand 

spectral matrix A of A, and verify that MA = AM 
Deduce that the system of difference equations 


x(k + 1) = Ax(k) 
where x(k) = [x,(k) x(k) x,(4)]', has a solution 
x(k) = My(h) 
where y(k) = A“p(0). Find this solution, given 
OST CO OF. 


The system shown in Figure 6.26 is a realization 
of a discrete-time system. Show that, with state 
variables x,(k) and x,(k) as shown, the system may 
be represented as 


Figure 6.26 Discrete-time system of Review 
exercise 19. 
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x(k + 1) = Ax(k) + bu(k) 
y(k) = c'x(k) 


Calculate the z transfer function of the system, 
D(z), where 


D(z) = e(zl- Ay'b 


Reduce the system to control canonical form by 
the following means: 


(i) calculate the controllability matrix M., where 
M.=[b Ad] is the matrix with columns b 
and Ab; 


(ii) show that rank (M)) = 2, and calculate M;'; 


(iii) write down the vector v' corresponding to 
the last row of M;'; 


(iv) form the matrix T= [v"’ v'AJ", the matrix 
with rows v' and v'A; 


(v) calculate T’ and using this matrix T, 
show that the transformation z(4) = Tx(k) 
produces the system 


2(k + 1) = TAT '2(k) + Thu(k) 
= Cz(A) + buh) 


where C is of the form 


al 


and b,=[0 1)’. Calculate a and B, and 
comment on the values obtained in relation 
to the transfer function D(z). 
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Introduction 


The representation of a function in the form of a series is fairly common practice in 
mathematics. Probably the most familiar expansions are power series of the form 


fiz) = Sia," 


n=0 
in which the resolved components or base set comprise the power functions 


|e Ae ae cee ee 
For example, we recall that the exponential function may be represented by the infinite 
series 

x x x" co x" 
e Slat ty teeta te= 
n=0 
There are frequently advantages in expanding a function in such a series, since the first 
few terms of a good approximation are easy to deal with. For example, term-by-term inte- 
gration or differentiation may be applied or suitable function approximations can be made. 
Power functions comprise only one example of a base set for the expansion of func- 

tions: a number of other base sets may be used. In particular, a Fourier series is an 
expansion of a periodic function f(f) of period T= 27/q in which the base set is the set 
of sine functions, giving an expanded representation of the form 


f(t) = Ag+ x A, sin(nat + ¢,) 


n=1 


Although the idea of expanding a function in the form of such a series had been used 
by Bernoulli, D’Alembert and Euler (c. 1750) to solve problems associated with the 
vibration of strings, it was Joseph Fourier (1768—1830) who developed the approach to 
a stage where it was generally useful. Fourier, a French physicist, was interested in 
heat-flow problems: given an initial temperature at all points of a region, he was con- 
cerned with determining the change in the temperature distribution over time. When 
Fourier postulated in 1807 that an arbitrary function f(x) could be represented by a 
trigonometric series of the form 


S (A,, cos nkx + B, sin nkx) 


n=0 


the result was considered so startling that it met considerable opposition from the 
leading mathematicians of the time, notably Laplace, Poisson and, more significantly, 
Lagrange, who is regarded as one of the greatest mathematicians of all time. They ques- 
tioned his work because of its lack of rigour, and it was probably this opposition that 
delayed the publication of Fourier’s work, his classic text Théorie Analytique de la 
Chaleur (The Analytical Theory of Heat) not appearing until 1822. This text has since 
become the source for the modern methods of solving practical problems associated 
with partial differential equations subject to prescribed boundary conditions. In addi- 
tion to heat flow, this class of problems includes structural vibrations, wave propagation 
and diffusion, which are discussed in Chapter 9. The task of giving Fourier’s work a 
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more rigorous mathematical underpinning was undertaken later by Dirichlet (c. 1830) 
and subsequently Riemann, his successor at the University of Géttingen. 

In addition to its use in solving boundary-value problems associated with partial 
differential equations, Fourier series analysis is central to many other applications in 
engineering. In Chapter 5 we saw how the frequency response of a dynamical system, 
modelled by a linear differential equation with constant coefficients, is readily determined 
and the role that it plays in both system analysis and design. In such cases the frequency 
response, being the steady-state response to a sinusoidal input signal A sin @t, is also a 
sinusoid having the same frequency as the input signal. As mentioned in Section 5.5.6, 
periodic functions, which are not purely sinusoidal, frequently occur as input signals in 
engineering applications, particularly in electrical engineering, since many electrical 
sources of practical value, such as electronic rectifiers, generate non-sinusoidal periodic 
waveforms. Fourier series provide the ideal framework for analysing the steady-state 
response to such periodic input signals, since they enable us to represent the signals as 
infinite sums of sinusoids. The steady-state response due to each sinusoid can then be 
determined as in Section 5.8, and, because of the linear character of the system, the 
desired steady-state response can be determined as the sum of the individual responses. 
As the Fourier series expansion will consist of sinusoids having frequencies n@ that are 
multiples of the input signal frequency @, the steady-state response will also have com- 
ponents having such frequencies. If one of the multiple frequencies n@ happens to be 
close in value to the natural oscillating frequency of the system, then it will resonate with 
the system, and the component at this frequency will dominate the steady-state response. 
Thus a distinction of significant practical interest between a non-sinusoidal periodic input 
signal and a sinusoidal input signal is that although the signal may have a frequency 
considerably lower than the natural frequency of the system, serious problems can still 
arise owing to resonance. A Fourier series analysis helps to identify such a possibility. 

In Chapter 8 we shall illustrate how Fourier series analysis may be extended to 
aperiodic functions by the use of Fourier transforms. The discrete versions of such 
transforms provide one of the most advanced methods for discrete signal analysis, 
and are widely used in such fields as communications theory and speech and image 
processing. Applications to boundary-value problems are considered in Chapter 9. 


Fourier series expansion 


In this section we develop the Fourier series expansion of periodic functions and dis- 
cuss how closely they approximate the functions. We also indicate how symmetrical 
properties of the function may be taken advantage of in order to reduce the amount 
of mathematical manipulation involved in determining the Fourier series. First the 
properties of periodic functions are briefly reviewed. 


Periodic functions 


A function f(A is said to be periodic if its image values are repeated at regular intervals 
in its domain. Thus the graph of a periodic function can be divided into ‘vertical strips’ 
that are replicas of each other, as illustrated in Figure 7.1. The interval between two 
successive replicas is called the period of the function. We therefore say that a function 


f(O is periodic with period 7 if, for all its domain values ¢, 
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Figure 7.1 A periodic 
function with period T. 


7.2.2 


fO, 


+(T—t;) 


| 
| : ! 
| | 
| I 


| 
= One period ——>—— One period — 


ft+ mT) =f) 


for any integer m. 
To provide a measure of the number of repetitions per unit of ¢, we define the frequency 
of a periodic function to be the reciprocal of its period, so that 


frequency = == 
7 y period 7 


The term circular frequency is also used in engineering, and is defined by 
circular frequency = 27 x frequency = 


and is measured in radians per second. It is common to drop the term ‘circular’ and refer 
to this simply as the frequency when the context is clear. 


Fourier’s theorem 


This theorem states that a periodic function that satisfies certain conditions can be 
expressed as the sum of a number of sine functions of different amplitudes, phases and 
periods. That is, if f(4) is a periodic function with period 7 then 


f(t) =A, + A, sin(@t+ 6,) + A, sinQ@t+ o,)+... 
+A,sin(no@t+ o,) +... (7.1) 


where the As and @s are constants and w = 2n/T is the frequency of f(t). The term 

A, sin(@t + @,) is called the first harmonic or the fundamental mode, and it has the 

same frequency @ as the parent function f(f). The term A, sin(n@t + @,) is called the 

nth harmonic, and it has frequency n@, which is n times that of the fundamental. 4, 

denotes the amplitude of the nth harmonic and @, is its phase angle, measuring the lag 

or lead of the nth harmonic with reference to a pure sine wave of the same frequency. 
Since 


A, sin(not + $,) = (A, cos @,)sin not + (A, sin @,,) cos nwt 
= 5b, sinnot + a, cos not 
where 
b, = A,,cos @,, a,= A, sin @, (7.2) 


the expansion (7.1) may be written as 
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fO= 5a a, COS not + > b, sin not (7.3) 


nl n=1 


where dy = 2A, (we shall see later that taking the first term as t do rather than ay is a 
convenience that enables us to make ay fit a general result). The expansion (7.3) is called 
the Fourier series expansion of the function f(4), and the as and bs are called the Fourier 
coefficients. In electrical engineering it is common practice to refer to a, and b, respect- 
ively as the in-phase and phase quadrature components of the mth harmonic, this 
terminology arising from the use of the phasor notation e" = cosnat + jsinnat. 
Clearly, (7.1) is an alternative representation of the Fourier series with the amplitude 
and phase of the nth harmonic being determined from (7.2) as 


A,=\(@2+B), d= tan'(#) 


with care being taken over choice of quadrant. 
The Fourier coefficients are given by 


d+T 
a, = | f(t) cos not dt (n=0,1,2,...) 4) 
d 
2 d+T 
| f(t) sin not dt (n= ie 2s 33233) (7.5) 
d 


which are known as Euler’s formulae. 
Before proceeding to verify (7.4) and (7.5), we first state the following integrals, in 
which T= 21n/@: 


d+T 

| cos nat dt = . tee) (7.6) 
5 T (n=0) 
d+T 

| sinn@tdt=0 (all n) (7.7) 
d 
s 0 (m#n) 

sin mot sin not dt = {ir Gane (7.8) 


0 (m 7 n) 
cos m@tcos nat dt = 7 9 
] 51 (m 17 0) ( ) 


d+T 
| cosmo@tsinnatdt=0 (all m and n) (7.10) 
The results (7.6)-(7.10) constitute the orthogonality relations for sine and cosine 
functions, and show that the set of functions 
{1, cos Mt, cos2@t,..., cosnmt, sin wt, sin2at,..., sinnar} 


is an orthogonal set of functions on the interval d < t < d+ T. The choice of d is 
arbitrary in these results, it only being necessary to integrate over a period of duration 7. 
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Integrating the series (7.3) with respect to ¢ over the period t= d to t=d+ T, and 
using (7.6) and (7.7), we find that each term on the right-hand side is zero except for 
the term involving a); that is, we have 


d+T d+T oe d+T d+T 
| fro.ar= joo] oF («,] cos mana 6,| sin nana 
n=1 


d d d d 
= 34(T) + [4,(0) +6,(0)] 
n=1 
= }Tay 


Thus 
1 d+T 
tay = i) f(t) dt 


and we can see that the constant term } a, in the Fourier series expansion represents the 
mean value of the function /(‘) over one period. For an electrical signal it represents the 
bias level or DC (direct current) component. Hence 
d+T 
a | f(t) dt (7.11) 
r d 

To obtain this result, we have assumed that term-by-term integration of the series (7.3) 
is permissible. This is indeed so because of the convergence properties of the series — 
its validity is discussed in detail in more advanced texts. 

To obtain the Fourier coefficient a, (n #0), we multiply (7.3) throughout by cos mat 
and integrate with respect to ¢ over the period t= d to t=d + T, giving 


d+T d+T es d+T 
| f(t) cos mat dt = | cos mat dt + by | cos nwt cos mat dt 


d d n=1 d 


on d+T 
+ > | cos mot sin nat dt 


n=1 d 


Assuming term-by-term integration to be possible, and using (7.6), (7.9) and (7.10), we 
find that, when m # 0, the only non-zero integral on the right-hand side is the one that 
occurs in the first summation when n = m. That is, we have 


d+T d+T 
| f(t) cos mat dt = | cos ma@t cos mat dt = }.a,,T 


d d 


giving 


d+T 
a= 2| f(t) cos not dt (7.12) 
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The value of a, given in (7.11) may be obtained by taking m = 0 in (7.12), so that we 
may write 


d+T 
d, = Al f()cosmatdt (n=0,1,2,...) 


d 


which verifies formula (7.4). This explains why the constant term in the Fourier series 
expansion was taken as ! a, and not ap, since this ensures compatibility of the results 
(7.11) and (7.12). Although a, and a, satisfy the same formula, it is usually safer to 
work them out separately. 

Finally, to obtain the Fourier coefficients b,, we multiply (7.3) throughout by 
sin mat and integrate with respect to ¢ over the period t= d to t=d + T, giving 


d+T d+T 
| f(t) sin mat dt = | sin mot dt 


d d 


d+T d+T 
+ >, .| sin mat cos not dt + ,| sin m@t sin not 7 


n=1 d t 


Assuming term-by-term integration to be possible, and using (7.7), (7.8) and (7.10), we 
find that the only non-zero integral on the right-hand side is the one that occurs in the 
second summation when m = n. That is, we have 


d+T d+T 
| f() sin mot dt = | sin mot sin mat dt = $b,,T 


m 
d d 


giving, on replacing m by n, 


a T 


d 


d+T 
b 2 f(sinnotdt (n=1,2,3,..-) 
which verifies formula (7.5). 


Summary 


In summary, we have shown that if a periodic function f(f) of period T = 2n/@ can 
be expanded as a Fourier series then that series is given by 


fO= 50+ sy a, COS not + oy b, sinnat (7.3) 


n=1 n=1 


where the coefficients are given by the Euler formulae 


d+T 

= 2] f@cosnotdt (n=0,1,2,...) WE) 
d 
d4+T 

i= 2] f(@) sinnotdt (n=1, 2,3,...) i) 
d 


www.2Ofile.org 


566 


FOURIER SERIES 


7.2.3 


The limits of integration in Euler’s formulae may be specified over any period, so that 
the choice of d is arbitrary, and may be made in such a way as to help in the calculation of 
a, and b,. In practice, it is common to specify f(¢) over either the period —} T< t < } Tor 
the period 0 < ¢ < T, leading respectively to the limits of integration being -} T and 
}T (that is, d =—}T) or 0 and T (that is, d = 0). 

It is also worth noting that an alternative approach may simplify the calculation of 
a, and b,. Using the formula 


e”" = cosnat + j sinnat 


we have 


d+T 
b= . | fer dt (7.13) 


d 


Evaluating this integral and equating real and imaginary parts on each side gives the 
values of a, and b,. This approach is particularly useful when only the amplitude 
|a,,+ jb,,| of the nth harmonic is required. 


Functions of period 2x 


If the period T of the periodic function f(‘) is taken to be 2m then @ = 1, and the 
series (7.3) becomes 


f= fat » a, cos nt + y b, sin nt (7.14) 
ial iI 
with the coefficients given by 
1 d+20 
oa 1 fOcsmeé: (=; I, 2 sas) (7.15) 
d 
1 d+20 
d= 1| (KO) stararck: (a= i, 2 oo) (7.16) 
d 


While a unit frequency may rarely be encountered in practice, consideration of this par- 
ticular case reduces the amount of mathematical manipulation involved in determining 
the coefficients a, and b,. Also, there is no loss of generality in considering this case, 
since if we have a function f(‘) of period 7, we may write t, = 2m7¢/T, so that 


fo =f(F3)= Fe) 


where F(t,) is a function of period 27. That is, by a simple change of variable, a periodic 
function f(f) of period T may be transformed into a periodic function F(t,) of period 21. 
Thus, in order to develop an initial understanding and to discuss some of the properties 
of Fourier series, we shall first consider functions of period 27, returning to functions 
of period other than 27 in Section 7.2.7. 
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Example 7.1 Obtain the Fourier series expansion of the periodic function f(t) of period 27 defined by 
f=t O<t<2n), fd =At+2n) 


Figure 7.2 Sawtooth fo 
wave of Example 7.1. 


Solution A sketch of the function /(f) over the interval —4m < t < 4m is shown in Figure 7.2. 
Since the function is periodic we only need to sketch it over one period, the pattern 
being repeated for other periods. Using (7.15) to evaluate the Fourier coefficients a) and 
a, gives 


20 Qn 2 20 
a= 2] rinar=2| care lS = 2n 
Tt Tt mt | 2 


0 0 0 


and 


g 
lI 


| f(cosntdt (n=1,2,...) 


1 20 
=- tcos nt dt 
7 0 


which, on integration by parts, gives 


ee | =1 (2% sin 2nn-+t cos ann £980) = 


2 
Tt n n n n n 


since sin 2mm = 0 and cos 2nmt = cos 0 = 1. Note the need to work out ay separately from 


a, in this case. The formula (7.16) for 6, gives 


20 
b= | f(sinntdt (n=1,2,...) 


0 


1 20 
= tsin nt dt 
Tt 


0 


which, on integration by parts, gives 


2 


1 2 20 
b, = =| -£ cos nt+ SH! 
Tw) n n 


0 


1(_22 cos 2nn) (since sin 2mm = sin 0 = 0) 
TL on 


=-= (since cos 2nm = 1) 
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Hence from (7.14) the Fourier series expansion of /(f) is 


co 


fj=n- > 2 sin nt 


n=1 


or, in expanded form, 


Ath=a- 2{ sin r+ Say snot pS } 
n 


Example 7.2 A periodic function f(t) with period 21 is defined by 
fatP+t (a<t<n), f=f(t+2n) 


Sketch a graph of the function f(A) for values of ¢ from ¢ = —3m to ¢ = 3m and obtain a 
Fourier series expansion of the function. 


Figure 7.3 Graph of fo 
the function f(7) of 
Example 7.2. 
1 
= t t + 
~3n -2n a O 2n 3n ¢ 


Solution A graph of the function f(t) for —-3m < t < 3m is shown in Figure 7.3. From (7.15) 
we have 


1 ‘ 1 i 2 2 
a= jar =2] (t +t)dt=2n 
and 


a=} flt)cosntdt (n=1,2,3,...) 


Tt 
| (f° +t) cos nt dt 
—T 
which, on integration by parts, gives 


1] 2 2 2 1 : 
ft. t : 7 
a, = — sinnt+ = cosnt- — sinnt+ — sinnt+ — cos nt 
Tl) n n n n n - 


Tv 
14 : : 
=> ails cosnm | since sinnn = 0 and a cosnt}| =0 
Tn n 


= 4(-1)" (since cos nt = (—1)") 
n 
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From (7.16) 


12 f(t)sinntdt (n=1,2,3,...) 
Tl 


Tw 
| (f +1) sin nt dt 
—T 


which, on integration by parts, gives 


Tw 
2. 
b,=- —pcos nt + 2b 6in nt+ 2 cos nt - Lcos nt + 4 sin nt 
T) on n n n n 


= ~2 00s nn = iy (since cos nm = (—1)") 
n n 
Hence from (7.14) the Fourier series expansion of f(¢) is 


- 4 n — 2 n . 
f(t) = $+ Y¥ A(-1)"cos nt - Y =(-1)"sin nt 
3 2 bi 2 n 
or, in expanded form, 
f(t) = 1h? + 4{—cos poe ee a. : .) + 2{sin r- Siu ae .) 
. 2 3? a) 3 


To illustrate the alternative approach, using (7.13) gives 


atib=2| f(tjedt = | (P+ e" dt 


T 


= (Se -| lena 
mT | jn in 


Tv 
: +o ine 2t+ Wine a 


jn (jn) (jn) |, 


Since 
e”"" = cosnt + jsinnn = (-1)" 
e""" = cosnt — j sinnn = (-1)" 
and 
| 


; -1)"( .wWtn 2n+1 j2. .w-n 1-20 }2 
a, + jb, =U (jE te, 28) ee 1 2e 2) 
T n n n n n n 
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Figure 7.4 Piecewise- 
continuous function 
over a period. 


Example 7.3 


Equating real and imaginary parts gives, as before, 


4 n 2 n 
= Sy Rae 
n n 


A periodic function f(t) may be specified in a piecewise fashion over a period, or, 
indeed, it may only be piecewise-continuous over a period, as illustrated in Figure 7.4. 
In order to calculate the Fourier coefficients in such cases, it is necessary to break up 
the range of integration in the Euler formulae to correspond to the various components 


of the function. For example, for the function shown in Figure 7.4, f(¢) is defined in the 
interval —m < t < 1 by 


A(t) (-m™<t<-p) 
SO=) At) (-p<t<q) 
BM) (q<t<m) 


and is periodic with period 2m. The Euler formulae (7.15) and (7.16) for the Fourier 
coefficients become 


- | fd cosmar+| f0 cosmar+| 


Tw 


(4) cos nt 7 


q 


- | fo snmars | f0 sinmar | 


1 


A3(4) sin nt er 


q 


A periodic function f(f) of period 27 is defined within the period 0 S t S 27 by 
t (0<t<jn) 
At) = 450 


m-3t ("<t< 2m) 


(in <t<n) 
Sketch a graph of f(A) for —2m < t S 3m and find a Fourier series expansion of it. 
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Figure 7.5 Graph of f() 
the function f(7) of 
Example 7.3. 
; - 
-2n — O 53x TX 2n 3n of 


Solution A graph of the function f(t) for —2” < t < 3m is shown in Figure 7.5. From (7.15), 


20 n/2 T 20 
a= | f(dr=4 | rar+| nar | (- 4t)dz| = sn 
0 0 n/2 T 


20 
a= 4 f(t) cosntdt (n=1,2,3,...) 


m/2 wT 2n 
1 
-1| reos tars | prcosmrar+| (nm - 32) cos nt dt 
T 
n/2 


0 2 qT 


n/2 T ) . on 
t cos ne . Pent 
sin nt+S™) 4) Esinne| + | eo! SOM - cosnt 
2 
n n 2n i 2 n on : 


0 


and 


Tt 1 1 1 1 
— sin 5nl + cos aes eames S| 5nT — = + == Cos nt 
n n 2n 2n? 2n? 


= (2¢0s int — 3 + cos nt) 


2mn 
that is, 
1 n/2 
st =) -1] (evenn) 
Tn 
a= 
ate (odd n) 
Tn 
From (7.16), 


bat] f(t)sinntdt (n=1,2,3,...) 


wv 2n 
: 1 : 
=" sinnears | ie sinncars | (= - 51) sin near 
m/2 


TT 


n/2 T 
~“cos nt+ sin n =F —L eos nt 
2n 
0 n/2 


l 20 
F cos N= 5 sin nt 
2n 2n° 


TT 


se 


L 
Tt 
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1 Tt Is Tt Tt Tt 
= =|-—cos int + — sin int - —cos nm + —cos int + —cos nt 


um\ 2n n 2n 2n 2n 
acu sin tnt 
= 2 2 
Tn 
0 (even 7) 
= fe (n-1)/2 
SS (odd n) 
Tn 


Hence from (7.14) the Fourier series expansion of f(t) is 


f(t) =21 - 2( cos t+ = + oT Hae } 
T ce 5? 
~ 2( S082 cos 6f , cos 10F 4 .) 
m\ 2 6 10 


37 5° qe 


+ A(sin ¢- sin 3¢ | sin 5t oe 
T 


(2) A major use of the MATLAB Symbolic Math Toolbox and MAPLE, when dealing with 
Fourier series, is to avoid the tedious and frequently error prone integration involved in 
determining the coefficients a, and b,,. It is therefore advisable to use them to check 
the accuracy of integration. To illustrate we shall consider Examples 7.2 and 7.3. 

In MAPLE n may be declared to be an integer using the command 


assume(n, integer) ; 


which helps with simplification of answers. There is no comparable command in 
MATLAB so, when using the Symbolic Math Toolbox, we shall use the command 
maple(‘assume (n,integer) ’) 
Considering Example 7.2 the MATLAB commands 
syms t n 
maple(‘assume (n,integer)’); 
ime (EZ a E) COS (iE) g=i9: 191.) /iow 
return the value of a, as 
AL? (AL) in fir 2 
El) 


2. 


, Where n~ indicates 


Entering the command pretty (ans) gives a, in the form 4 
that 7 is an integer. Likewise the commands 
dime ((E*2 se (2) "Salina (ia ie) =i , Oi.) fel ¢ 
pretty (ans) 


return b, as 


thus checking with the values given in the solution. 
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The corresponding commands in MAPLE are 


assume (n,integer) ; 
sate ((Ce2 42 je) Cros (iaeic) , it = —Pa, Pa) (Pate 
returning the value of a, as 
(i) 
mene 


with the further command 
dime (ier? Te) esac), i = Sia, . ia.) eats 


4 


returning the value of 5, as 
(=a)" 
n~ 
again checking with the values given in the solution. 
In Example 7.3 we are dealing with a piecewise function, which can be specified 
using the piecewise command. In MATLAB the commands 


=e 


syms t n 
maple(‘assume (n,integer)'); 
£ = (“PIECEWISE ([t,t<= 1/2*pi)),  [l/2*pr, 1/2*pi—t<= 0) and 


=< = 0], leieL/ Pest, w1<Sse]))) °) 2 
aime (GE COS (AE) , O, Baa) TO. 2 
pretty (ans 


return the value of a, as 
1/92 SiR pi me) + (=2) 
n~ pi 


with the further commands 


sialic (Ge Sala (ial }) , O), 2 ioaL)) isk 2 
pretty (ans) 


returning the value of 5, as 
sin(1/2 pi n~) 


n~ pi 
In MAPLE the commands 
f:= simplify (piecewise(t<= Pi/2,t,(t>= Pi/2 and 
= Pi), Pi/2,ts= Pi, Pic /2))) 9 
Fi S= jwiaeljoro ly (iE, 1c) § 
assume (n,integer) ; 
eidte= simie (Je: (ie) eos acc), © = 0. . Ps) (Pup 
Iga = abate (VEE (ie) Samn(iatc)) , t = OW. . eu) (Pas 


return the same values as MATLAB above for a, and 6,,. 
An alternative approach to using the piecewise command is to express the 
function in terms of Heavyside functions. 


7.2.4 Even and odd functions 


Noting that a particular function possesses certain symmetrical properties enables us 
both to tell which terms are absent from a Fourier series expansion of the function and 
to simplify the expressions determining the remaining coefficients. In this section we 
consider even and odd function symmetries. 
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y 


Figure 7.6 Graphs of 
(a) an even function 
and (b) an odd 
function. 


First we review the properties of even and odd functions that are useful for deter- 
mining the Fourier coefficients. If f(f) is an even function then f(t) = f(—A) for all t, and 
the graph of the function is symmetrical about the vertical axis as illustrated in Figure 
7.6(a). From the definition of integration, it follows that if /() is an even function then 


| fit) dt = 2 | fit) dt 


0 


If f(4) is an odd function then f(t) = —f(—4) for all f, and the graph of the function is 
symmetrical about the origin; that is, there is opposite-quadrant symmetry, as illustrated 
in Figure 7.6(b). It follows that if f(#) is an odd function then 


| ft) dt = 0 


The following properties of even and odd functions are also useful for our purposes: 
(a) | the sum of two (or more) odd functions is an odd function; 

(b) the product of two even functions is an even function; 

(c) the product of two odd functions is an even function; 

(d) the product of an odd and an even function is an odd function; 

(e) the derivative of an even function is an odd function; 

(f) the derivative of an odd function is an even function. 


(Noting that f°" is even and ¢°“is odd helps one to remember (a)—(f).) 
Using these properties, and taking d=— 5 Tin (7.11) and (7.12), we have the following: 


(i) ‘If, f( is an even periodic function of period 7 then 


T/2 T/2 
a, = 2| fi) cos not dt = | f(t) cos not dt 
0 


-T/2 
using property (b), and 
T/2 
bee | f (0) sin net dt = 0 
r -T/2 
using property (d). 


Thus the Fourier series expansion of an even periodic function f(t) with period 
T consists of cosine terms only and, from (7.3), is given by 


S() = 340+ Ya, cos not (7.17) 
n=1 
with 
inp 
a= | fcosnoat (n=0,1,2,...) (7.18) 
0 
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(ii) If f(#) is an odd periodic function of period 7 then 


T/2 
a, = 2| f(é) cos not dt = 0 


-T/2 
using property (d), and 
T/2 T/2 
p=2 | f(Osinnotdt = 4 | {( sin nat dt 
r -T/2 r 0 
using property (c). 


Thus the Fourier series expansion of an odd periodic function f(¢) with period T 
consists of sine terms only and, from (7.3), is given by 


A) = ¥ by sinneot (7.19) 


n=1 


with 


b, fd)sinnotdt (n=1,2,3,...) (7.20) 


lI 
TES 
S =) 


Example 7.4 A periodic function /(#) with period 2x is defined within the period -m < t < 1 by 


_fJ-l (-1t<t<0) 
fo = 1 (0<t<n) 


Find its Fourier series expansion. 


Figure 7.7 Square f(t) 
wave of Example 7.4. 


+ — 
(4x [-3n j-2ny-n |O jm 42m [3m [4m 7 

' ! 1 I I I I 

a ae — 


Solution A sketch of the function /(f) over the interval —-4m < t < 4m is shown in Figure 7.7. 
Clearly f(¢) is an odd function of ¢, so that its Fourier series expansion consists of sine 
terms only. Taking 7 = 27, that is @= 1, in (7.19) and (7.20), the Fourier series expan- 
sion is given by 


fi) = Yb, sinnt 
n=1 


with 
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be 2| J(@sinntdt (n=1,2,3,...) 
Tt 
0 


1 sinntdt = 2)_1 cos nt 
0 ut n 0 


= 24 — cosnt) = afi —(-1)"] 
nt nm 
_ 4/nm (odd n) 
0 (even n) 


Thus the Fourier series expansion of f(f) is 


fA) = 7 (sin + tsi 344 !sin 54+...) = 2p BEB Dt (7.21) 
T : > T 2n-1 


n=1 
Example 7.5 _A periodic function f(t) with period 27 is defined as 


fQ=P? (a<t<m,  f=flt+2n) 


Obtain a Fourier series expansion for it. 


Solution A sketch of the function f(7) over the interval —3n < ¢ < 3m is shown in Figure 7.8. 
Clearly, f(t) is an even function of f, so that its Fourier series expansion consists of 
cosine terms only. Taking 7 = 27, that is @ = 1, in (7.17) and (7.18), the Fourier series 
expansion is given by 


f= fat Dy a, COs nt 
n=1 


with 


a= 2 foar=2| P dt= in 
Tl Tl A 


Figure 7.8 The fOd 
function f(t) of 
Example 7.5. 
3x -2n -n O a 2x 3x t 
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and 


f(cosntdt (n=1,2,3,...) 


a ity 


2, 
f cosntdt = a|¢ sin nt + 24 cos nt- 2 sin nt 
Tin 


nN n 


a, = 


| 
| 


a ity 


0 
= 2 ( 2 cos nn) = iy 
T\ nN nN 
since sin = 0 and cosnn = (—1)". Thus the Fourier series expansion of f(¢) = f° is 


f= Ly? +4 BY Cl cos nt (7.22) 
n=1 n 


or, writing out the first few terms, 


es Die? 4 
S() = 3@ — 4 cost + cos2t—5cos3tt+... 


7.2.5 Linearity property 


The linearity property as applied to Fourier series may be stated in the form of the 
following theorem. 


Theorem 7.1 If f(¢) =/g(t) + mh(t), where g(#) and h(f) are periodic functions of period T and / and m 


are arbitrary constants, then f(t) has a Fourier series expansion in which the coefficients 
are the sums of the coefficients in the Fourier series expansions of g(f) and h(t) multi- 
plied by / and m respectively. 


Proof Clearly f(t) is periodic with period T. If the Fourier series expansions of g(t) and A(f) are 


g()=ta,+ by a, COS N@t + py b, sinnot 


n=1 n=1 


h(t) =} O%q+ y OL, COS N@t + 3 B, sinnot 


n=1 n=1 


then, using (7.4) and (7.5), the Fourier coefficients in the expansion of f(t) are 


d+T d+T 
A,= 2| f(t) cos not dt = 2| [/g(t) + mh(t)] cosnat dt 


d d 


d+T d+T 
= x| @(t) cos nat dt + 2a | A(d) cos noot dt = la, + ma, 


d d 


www.2Ofile.org 


578 FOURIER SERIES 


and 


d+T 


d+T d+T 
B32 | f(Osinnot dt = 2 | o(f) sin not dt + m | A(t) sin noot dt 


d d 


= lb, + mB, 
confirming that the Fourier series expansion of f(f) is 


f(t) = § (lay + m&%) + > (la, + ma) cos nwt + 3 (lb, + mB,) sin nat 


n=1 n=1 


end of theorem 


Example 7.6 Suppose that g(f) and A(f) are periodic functions of period 27 and are defined within the 
period -t < t< 1 by 


a=", AO=t 


Determine the Fourier series expansions of both g(t) and A(t) and use the linearity 
property to confirm the expansion obtained in Example 7.2 for the periodic function f(t) 
defined within the period -n < t< nbyf()=f? +t. 


Solution The Fourier series of g(f) is given by (7.22) as 
g(t) =1n eS Cl cos nt 
n=1 n 


Recognizing that A(t) = ¢ is an odd function of t, we find, taking 7 = 27 and w= 1 in 
(7.19) and (7.20), that its Fourier series expansion is 


h(t) = b, sin nt 


n=1 


where 


Sie 


= A(t)sinntdt (n=1,2,3,...) 


‘ Tt 
-2 “sin n= oles cos nt + SBM 
Tt n n 


2: n 
-*(-1) 
n 
recognizing again that cos nm = (—1)" and sin nz = 0. Thus the Fourier series expansion 
of h(t) = tis 
h(t) =-2>. eat (7.23) 
n=1 ut 


www.2Ofile.org 


7.2 FOURIER SERIES EXPANSION 579 


Using the linearity property, we find, by combining (7.22) and (7.23), that the Fourier 
series expansion of f(f) = g(f) + A() = t? + tis 


co =| n co =] n . 
f(t) =1w +4 ly cones al) ina 
3 py an 2 n 


which conforms to the series obtained in Example 7.2. 


7.2.6 Exercises 


Check evaluation of the integrals using MATLAB or MAPLE whenever possible. 


In each of the following a periodic function f(t) of 3 The charge q(¢) on the plates of a capacitor at time 
period 27 is specified over one period. In each case tis as shown in Figure 7.9. Express q(t) as a Fourier 
sketch a graph of the function for —4n S ¢ S 4 and series expansion. 


obtain a Fourier series representation of the function. 


(a) f(z) -\* (—t<t<0) _ 
t (0<t<m) : 
_jttm (-m<t<0) 
(b) f(t) -| 0 (0<t<n) On —- O n 2x 3m 4a t 


) fO=1 t Wein Figure 7.9 Plot of the charge g(t) in Exercise 3. 
c) f)=1-- <t<2n 
(©) . 3 ) 


0 (-t St S-5n) 4 The clipped response of a half-wave rectifier is the 
) fi) =42 cost (nm <1 <!n) periodic function f() of period 27 defined over 
: 2 the period 0 < t < 2m by 
0 Gn <t<n) 
fa) = 5sint¢ (0St<7z) 
(ec) f@=cos5t (m<t<7) 0 (n<t<2n) 
@) fO=l4 Ca<t<n) Express (7) as a Fourier series expansion. 
(g) f(D = 0 eit) 5 Show that the Fourier series representing the 
2t-nm (0O<tS7) periodic function f(4), where 
(h) f(t) “ite ie eS) S(t) -| wm (-m<t<0) 
tte’ (0<t<n) (t-ny (0<t<n) 


Obtain the Fourier series expansion of the periodic S(t + 20) =f 
function f(t) of period 27 defined over the 
period 0 St S 2m by 


f=(n-t? (<t<2n) f= +> 2 cos nt+ x sin nt 
Use the Fourier series to show that nat [7 @ 

ee y (=1)""" _ 4 sinQ2n-1)t 

= n=1 n Tt n=l (2n ~ Sg 
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Use this result to show that Find the Fourier series expansion that represents the 


even function for all values of ¢, and use it to show that 


2 12 12_ 1 
= n= gh = 
n=l 1 8 2 (2n _ iy 
6 A periodic function /(¢) of period 21 is defined 7 A periodic function f(t) of period 27 is defined 
within the domain 0 < t < 1 by within the period 0 < t < 2m by 


t 


fo=| 


Sketch a graph of f(t) for —-2n < t < 4m for the two 


cases where 


(0 <¢< jn) 


fo=| 


2-t/k (OStS7T) 
t/T (m7 StS 2n) 


Draw a graph of the function for —4m < ¢ S 40 
and obtain its Fourier series expansion. 


By replacing ¢ by ¢ — } 7 in your answer, 
4 3 


Gnu <t<n) 


(a) f(2) is an even function show that the periodic function f(t — itt) —5 is 


(b) f(2) is an odd 


7.2.7 


Example 7.7 


Figure 7.10 
The function f(t) 
of Example 7.7. 


Solution 


2 
function represented by a sine series of odd harmonics. 


Functions of period T 


Although all the results have been related to periodic functions having period T, all the 
examples we have considered so far have involved periodic functions of period 27. This 
was done primarily for ease of manipulation in determining the Fourier coefficients 
while becoming acquainted with Fourier series. As mentioned in Section 7.2.3, functions 
having unit frequency (that is, of period 27) are rarely encountered in practice, and in 
this section we consider examples of periodic functions having periods other than 27. 


A periodic function f(t) of period 4 (that is, f(¢ + 4) = f(t) is defined in the range 
—2<t<2by 


O (2<7=20) 


no={ 
i @w2720) 


Sketch a graph of f(t) for —-6 < ¢ < 6 and obtain a Fourier series expansion for the 
function. 


A graph of f(¢) for —6 S t < 6 is shown in Figure 7.10. Taking 7'= 4 in (7.4) and (7.5), 
we have 


a=4] roa=f | oars | 1ar]=1 


www.2Ofile.org 


7.2 FOURIER SERIES EXPANSION 581 


anf f(t) cos $ tnnatdt (n=1,2,3,...) 
{| oars | costa | =0 


fo sin + int dt (n=1,2,3,...) 
=! “IT oars | sinjomrar|= Ect ~cosmm) = 2501-0) 
= ATM nT 
1 1 n 
= —(1- cosvm)=—[1-(-1) ] 
nT AT 
-{ 0 (even n) 


2/nm (odd n) 


Thus, from (7.3), the Fourier series expansion of f(f) is 


f(t)=3+ = (sin je tsin }ne+ tsin?mt+...) 


=e Ds 


7 ain 4(2n -1)mt 


Example 7.8 A periodic function /(¢) of period 2 is defined by 
3t (O<t<1 
fyi 
3 (1<t<2) 
fE+2)=fO 
Sketch a graph of f(t) for —4 < t < 4 and determine a Fourier series expansion for the 


function. 


Figure 7.11 fy 
The function /(4) 


of Example 7.8. 7 3 
| { I ! 
— 7 {$e 
3.4 t 


-4 3-2 -1 O 1 2 


Solution A graph of f(t) for—4 < t S 4 is shown in Figure 7.11. Taking T= 2 in (7.4) and (7.5), 
we have 


2 1 2 
=| foar= | sare | 3dt= 3 
0 0 1 


www.2Ofile.org 


582 FOURIER SERIES 


2 
a3] f(t) cos dt (n=1,2,3,...) 


0 


1 2 ; ; 
=| 3t cos mavars| 3 cos nat dt = aa SIRE RS USELUd a of. 


P 2 
3 sin nit 
2 
nt (nt) |, nt 


0 1 1 


0 (even n) 
-6/(nt) (odd n) 


=22 5(cosnt - b=| 
(nT) 


and 


2 
6. =3] f(t) sin dt (n= 1, 2,3,...) 


0 


1 2 
-| srsinmnrar | 3 sin nit dt 


0 1 


7 1 2 
_|_3cosnmt  3sinnmt) | 3cosnmt} __ 3 ae 
nt nt nt 


nt (nny |, 


Thus, from (7.3), the Fourier series expansion of f(f) is 


6 1 1 
f(t) =} - 5(cos nt + 5 cos 3ut+ >,cos Satt+...) 
T 


- 3 (sin nt-+!sin 2n¢-+!sin 3n¢+. ned 
1 


_9_ 6 cos(2n-1)nt_ 3 sinntt 
: > (2n-1)° > n 


Example 7.9 = Obtain the Fourier series expansion of the rectified sine wave 


f(D = |sin ¢| 


Solution A sketch of the wave over the interval —-m < t < 27 is shown in Figure 7.12. Clearly, 
f(t) is periodic with period x. Taking T = 7, that is, @ = 2, in (7.3)-(7.5) the Fourier 
series expansion is given by 


S(t) = bag + by a, cos 2nt 


n=1 


a= 2| sint dt = 4 
TY}, T 


Figure 7.12 Rectified fp 
wave f(t) =|sin¢|. 
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sin t cos 2nt dt 


2n+1 


1} 1 _t 
TI\2n+1 2n-1 


A sonnet cos2(n - ney 
T 


leer 
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[sin(2n + 1)t— sin(2n — 1)f] dt 


2n-1 


0 


1 - 1 
2n-1 


Jz 
m™ An*-1 


Thus the Fourier series expansion of f(f) is 


f=2-45 


T Tir 4n 


cos 2nt 
-1 


or, writing out the first few terms, 


f= = - £ (Jeos 2t+ tcos 4t+ 


7.2.8 Exercises 


Find a Fourier series expansion of the periodic 
function 


fd=t l<t<JI 
S(t +21) =f 


9 A periodic function f(A) of period 2/ is defined over 
one period by 
RU+0) (-1<1<0) 
ft) = 7B 
K 
Fup (0<t</) 


Determine its Fourier series expansion and illustrate 
graphically for —-3/ < t < 3/. 


10 A periodic function of period 10 is defined within 


the period -5 < t < 5 by 


no={° (-5 <1<0) 
3 (0<1t<5) 


13 


Determine its Fourier series expansion and illustrate 
graphically for -12 <t< 12. 


Lal 


Passing a sinusoidal voltage A sin wf through a 
half-wave rectifier produces the clipped sine wave 
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shown in Figure 7.13. Determine a Fourier series 
expansion of the rectified wave. 


fO 
A 


0) 


-2n/a  -n/w Nia  2n/lw 3nlw 


Figure 7.13 Rectified sine wave of Exercise 11. 


Obtain a Fourier series expansion of the periodic 
function 


f= (T<t<T) 
f(t + 2T) =f 
and illustrate graphically for —37 < ¢ < 37. 


Determine a Fourier series representation of the 
periodic voltage e(¢) shown in Figure 7.14. 


e(t) 


Figure 7.14 Voltage e(t) of Exercise 13. 
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7.2.9 


Theorem 7.2 


Convergence of the Fourier series 


So far we have concentrated our attention on determining the Fourier series expan- 
sion corresponding to a given periodic function f(t). In reality, this is an exercise in 
integration, since we merely have to compute the coefficients a, and b, using Euler’s 
formulae (7.4) and (7.5) and then substitute these values into (7.3). We have not yet 
considered the question of whether or not the Fourier series thus obtained is a valid 
representation of the periodic function /(¢). It should not be assumed that the existence 
of the coefficients a, and b,, in itself implies that the associated series converges to the 
function f(t). 

A full discussion of the convergence of a Fourier series is beyond the scope of 
this book and we shall confine ourselves to simply stating a set of conditions which 
ensures that f(t) has a convergent Fourier series expansion. These conditions, known as 
Dirichlet’s conditions, may be stated in the form of Theorem 7.2. 


Dirichlet’s conditions 

If f(t) is a bounded periodic function that in any period has 
(a) a finite number of isolated maxima and minima, and 
(b) a finite number of points of finite discontinuity 


then the Fourier series expansion of f(t) converges to f(f) at all points where /f(f) is 
continuous and to the average of the right- and left-hand limits of f(t) at points where 


f(é is discontinuous (that is, to the mean of the discontinuity). 


Example 7.10 


Solution 


end of theorem 


Give reasons why the functions 
1 . 1 
eee b rae 
@ = &) sin(—5) 


do not satisfy Dirichlet’s conditions in the interval 0 < ¢t < 27. 


(a) The function f(A = 1/(3 — 4) has an infinite discontinuity at t= 3, which is within 
the interval, and therefore does not satisfy the condition that f(#) must only have 
finite discontinuities within a period (that is, it is bounded). 


(b) The function f(4) = sin[1/(t — 2)] has an infinite number of maxima and minima 
in the neighbourbood of t = 2, which is within the interval, and therefore does not 
satisfy the requirement that /(‘) must have only a finite number of isolated 
maxima and minima within one period. 


The conditions of Theorem 7.2 are sufficient to ensure that a representative Fourier 
series expansion of f(f) exists. However, they are not necessary conditions for convergence, 
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and it does not follow that a representative Fourier series does not exist if they are not 
satisfied. Indeed, necessary conditions on f(t) for the existence of a convergent Fourier 
series are not yet known. In practice, this does not cause any problems, since for almost 
all conceivable practical applications the functions that are encountered satisfy the 
conditions of Theorem 7.2 and therefore have representative Fourier series. 

Another issue of importance in practical applications is the rate of convergence of 
a Fourier series, since this is an indication of how many terms must be taken in the 
expansion in order to obtain a realistic approximation to the function /(¢) it represents. 
Obviously, this is determined by the coefficients a, and b, of the Fourier series and the 
manner in which these decrease as n increases. 

In an example, such as Example 7.1, in which the function f(t) is only piecewise- 
continuous, exhibiting jump discontinuities, the Fourier coefficients decrease as |/n, 
and it may be necessary to include a large number of terms to obtain an adequate 
approximation to f(t). In an example, such as Example 7.3, in which the function 
is a continuous function but has discontinuous first derivatives (owing to the sharp 
corners), the Fourier coefficients decrease as 1/n’, and so one would expect the series 
to converge more rapidly. Indeed, this argument applies in general, and we may 
summarize as follows: 


(a) if f(#) is only piecewise-continuous then the coefficients in its Fourier series 
representation decrease as 1/n; 


(b) if f(A is continuous everywhere but has discontinuous first derivatives then the 
coefficients in its Fourier series representation decrease as 1/n’; 


(c) 1f f(é and all its derivatives up to that of the rth order are continuous but the 
(r + 1)th derivative is discontinuous then the coefficients in its Fourier series 
representation decrease as 1/n’”. 


These observations are not surprising, since they simply tell us that the smoother the 
function f(t), the more rapidly will its Fourier series representation converge. 

To illustrate some of these issues related to convergence we return to Example 7.4, 
in which the Fourier series (7.21) was obtained as a representation of the square wave 
of Figure 7.7. 

Since (7.21) is an infinite series, it is clearly not possible to plot a graph of the result. 
However, by considering finite partial sums, it is possible to plot graphs of approxima- 
tions to the series. Denoting the sum of the first N terms in the infinite series by fy (4), 
that is 


_4— sin(2n- 1)t 
IA) =D, ed (7.24) 


the graphs of f(t) for N = 1, 2, 3 and 20 are as shown in Figure 7.15. It can be seen 
that at points where f(f) is continuous the approximation of f(t) by f,(4) improves as 
N increases, confirming that the series converges to f(f) at all such points. It can also 
be seen that at points of discontinuity of (4), which occur at f=+nnm (n= 0, 1, 2,...), 
the series converges to the mean value of the discontinuity, which in this particular 
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Ai® 


Figure 7.15 Plots of f(t) for a square wave: (a) N = 1; (b) 2; (c) 3; (d) 20. 


example is ‘Cl + 1) =0. As a consequence, the equality sign in (7.21) needs to be 
interpreted carefully. Although such use may be acceptable, in the sense that the series 
converges to f(t) for values of tf where f(f) is continuous, this is not so at points of 
discontinuity. To overcome this problem, the symbol ~ (read as ‘behaves as’ or ‘repre- 
sented by’) rather than = is frequently used in the Fourier series representation of a 
function f(t), so that (7.21) is often written as 


In Section 7.7.3 it is shown that the Fourier series converges to f(t) in the sense that the 
integral of the square of the difference between f(t) and f,() is minimized and tends to 
zero as N > , 

We note that convergence of the Fourier series is slowest near a point of discontinu- 
ity, such as the one that occurs at t= 0. Although the series does converge to the mean 
value of the discontinuity (namely zero) at t= 0, there is, as indicated in Figure 7.15(d), 
an undershoot at t = 0° (that is, just to the left of = 0) and an overshoot at t = 0° (that 
is, just to the right of t= 0). This non-smooth convergence of the Fourier series leading 
to the occurrence of an undershoot and an overshoot at points of discontinuity of /(Z) is 
a characteristic of all Fourier series representing discontinuous functions, not only that 
of the square wave of Example 7.4, and is known as Gibbs’ phenomenon after the 
American physicist J. W. Gibbs (1839-1903). The magnitude of the undershoot/over- 
shoot does not diminish as V > © in (7.24), but simply gets ‘sharper’ and ‘sharper’, 
tending to a spike. In general, the magnitude of the undershoot and overshoot together 
amount to about 18% of the magnitude of the discontinuity (that is, the difference in the 
values of the function f(4) to the left and right of the discontinuity). It is important that 
the existence of this phenomenon be recognized, since in certain practical applications 
these spikes at discontinuities have to be suppressed by using appropriate smoothing 
factors. 
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To reproduce the plots of Figure 7.15 and see how the series converges as N 
increases use the following MATLAB commands: 


=o / LOO [SOOs S00] 

cals 

He|=\sjon Asis =L2ejou, <josl jou © © jou jou Brion Bos, Seo] 9 
wel=i = 2 a =i =h 2 2 =) =1 1 Ais 


foe mais 20 
ib aeaeral /joyak stale (( (Atal — 10 )) te) // (tal 1b) 6 
POLGIE (7, pt, L=Soe Soa, (0,01, R=", 10,0], l=1.3 1.53], 7r=") 


axis ([-3*pi,3*pi,-inf,inf]),pause 
end 


The pause command has been included to give you an opportunity to view the 
plots at the end of each step. Press any key to proceed. 


Theoretically, we can use the series (7.21) to obtain an approximation to 7. This is 
achieved by taking t= } =, when f(t) = 1; (7.21) then gives 


4x sin}(2n-1)n 


aio on-1 


n=1 


leading to 
oo n+l 
m=A4(1-}4}-14...)=4y5 EU 
ne 


For practical purposes, however, this is not a good way of obtaining an approximation 
to mt, because of the slow rate of convergence of the series. 


Functions defined over a finite interval 


One of the requirements of Fourier’s theorem is that the function to be expanded be 
periodic. Therefore a function f(¢) that is not periodic cannot have a Fourier series 
representation that converges to it for all values of t. However, we can obtain a Fourier 
series expansion that represents a non-periodic function f(¢) that is defined only over 
a finite time interval 0 < ¢t S Tt. This is a facility that is frequently used to solve 
problems in practice, particularly boundary-value problems involving partial dif- 
ferential equations, such as the consideration of heat flow along a bar or the vibrations 
of a string. Various forms of Fourier series representations of f(f), valid only in the 
interval 0 < t S f, are possible, including series consisting of cosine terms only or 
series consisting of sine terms only. To obtain these, various periodic extensions of f(f) 
are formulated. 


Full-range series 


Suppose the given function /(¢) is defined only over the finite time interval 0 S ¢ S T. 
Then, to obtain a full-range Fourier series representation of f(t) (that is, a series 
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consisting of both cosine and sine terms), we define the periodic extension (7) of 


S(t) by 


Figure 7.16 Graphs of 
a function defined only 
over (a) a finite interval 
0 <tS Tand (b) its 
periodic extension. 


Example 7.11 


Solution 


Figure 7.17 
The functions f(t) and 
o(t) of Example 7.11. 


oO=fQM O<t<77 
ot + D= o(/ 


The graphs of a possible f(#) and its periodic extension @(f) are shown in Figures 7.16(a) 
and (b) respectively. 

Provided that f(t) satisfies Dirichlet’s conditions in the interval 0 < ¢ S Tf, the 
new function $(f), of period t, will have a convergent Fourier series expansion. 
Since, within the particular period 0 < t < T, @(f) is identical with f(d), it follows 
that this Fourier series expansion of $(f) will be representative of f(4) within this 
interval. 


Find a full-range Fourier series expansion of f(¢) = ¢ valid in the finite interval 0 < t < 4. 
Draw graphs of both f(t) and the periodic function represented by the Fourier series 
obtained. 


Define the periodic function @(f) by 
oH=fH=t O<t<A4) 
o(t+ 4) = o@) 


Then the graphs of f(4) and its periodic extension @(¢) are as shown in Figures 7.17(a) 
and (b) respectively. Since @(f) is a periodic function with period 4, it has a convergent 
Fourier series expansion. Taking T = 4 in (7.4) and (7.5), the Fourier coefficients are 
determined as 


4 4 
«=| foar=4| tdt=4 


0 0 
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4 
a, = | f(t) cosinntdt (n=1,2,3,...) 


0 


4 
cos aa =0 
0 


4 
7 1 t cos $ntt dt = | sin inmt + 


2 


nt (nn) 


and 


4 
b, = | f@)sininntdt (=1,2,3,...) 


0 


4 4 
=+| tsinSnautdt = 5 _2t cos Snttt + 4 sininnt| = = 
0 nt (nt) ( 


Thus, by (7.3), the Fourier series expansion of @(f) is 


4. : : : : 
(ft) =2- ~(sin sat + sin ne+ ;sin yur + jsin 2¢+ ¢sin ant+...) 


4c 1. 
=2--Y -sininat 
“Dae 2m 


n=1 


Since @(f) = f(t) for 0 < t < 4, it follows that this Fourier series is representative of f(t) 
within this interval, so that 


fit)=t=2-2 5 tsintnnt (0<1 <4) (7.25) 
Tn 

It is important to appreciate that this series converges to ¢ only within the interval 

0 < t< 4. For values of ¢ outside this interval it converges to the periodic extended 

function @(¢). Again convergence is to be interpreted in the sense of Theorem 7.2, so 

that at the end points ¢ = 0 and t= 4 the series does not converge to ¢ but to the mean 

of the discontinuity in @(f), namely the value 2. 


Half-range cosine and sine series 


Rather than develop the periodic extension @(f) of f(4) as in Section 7.3.1, it is possible 
to formulate periodic extensions that are either even or odd functions, so that the result- 
ing Fourier series of the extended periodic functions consist either of cosine terms only 
or sine terms only. 

For a function f(f) defined only over the finite interval 0 < ¢t S T its even periodic 
extension F(t) is the even periodic function defined by 


ft) (0<t<7) 


F(t) =| 
f-t)) (-t<1t<0) 


F(t+ 27) =f(0 
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Figure 7.18 

(a) A function f(A); 
(b) its even periodic 
extension F(f). 


Figure 7.19 

(a) A function f(A); 
(b) its odd periodic 
extension G(f). 


As an illustration, the even periodic extension F(t) of the function f(t) shown in 
Figure 7.16(a) (redrawn in Figure 4.18(a)) is shown in Figure 4.18(b). 


f(t) F(t) 


Provided that f(f) satisfies Dirichlet’s conditions in the interval 0 < t < f, since it is 
an even function of period 27, it follows from Section 7.2.4 that the even periodic 
extension F(f) will have a convergent Fourier series representation consisting of cosine 
terms only and given by 


F(t) =1a,+ » a, COS a (7.26) 
where 
T 
| f(a) cos ar (2 Onieoe ae) vy) 
0 


Since, within the particular interval 0 < t < T, F(A) is identical with /(#), it follows that 
the series (7.26) also converges to f(¢) within this interval. 

For a function f(t) defined only over the finite interval 0 < ¢ S f, its odd periodic 
extension G(f) is the odd periodic function defined by 
ft) (0<t<9 


G(t) -| 
-f(-t) (-t<1t<0) 


G(t + 20) = Gt) 


Again, as an illustration, the odd periodic extension G(f) of the function f(‘) shown in 
Figure 7.16(a) (redrawn in Figure 7.19(a)) is shown in Figure 7.19(b). 


(a) 


Provided that f() satisfies Dirichlet’s conditions in the interval 0 < ft < T, since it is 
an odd function of period 27, it follows from Section 7.2.4 that the odd periodic exten- 
sion G(f) will have a convergent Fourier series representation consisting of sine terms 
only and given by 


GHe y best aa (7.28) 
pall 
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Solution 
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| fit) sin“ dt (n=1,2,3,...) (7.29) 


Again, since, within the particular interval 0 < t < 1, G(f) is identical with f(A), it 
follows that the series (7.28) also converges to f(¢) within this interval. 

We note that both the even and odd periodic extensions F(t) and G(f) are of period 
27, which is twice the length of the interval over which /f(f) is defined. However, the 
resulting Fourier series (7.26) and (7.28) are based only on the function /(4), and for this 
reason are called the half-range Fourier series expansions of /(¢). In particular, the 
even half-range expansion F(t), (7.26), is called the half-range cosine series expan- 
sion of f(¢), while the odd half-range expansion G(f), (7.28), is called the half-range 
sine series expansion of /(f). 


For the function f(t) = t defined only in the interval 0 < ¢ < 4, and considered in 
Example 7.11, obtain 


(a) a half-range cosine series expansion 


(b) a half-range sine series expansion. 


Draw graphs of f(4) and of the periodic functions represented by the two series obtained 
for -20 < t < 20. 


(a) Half-range cosine series. Define the periodic function F(t) by 


Fo =| f=t (0<t<A4) 
A-tH=-t (-4<t<0) 
F(t+ 8) = F(t) 


Then, since F(t) is an even periodic function with period 8, it has a convergent 
Fourier series expansion given by (7.26). Taking t= 4 in (7.27), we have 


4 4 
a= 3 oar =| tdt=4 


0 0 


4 
= :| f(t) cosinntdt (n=1,2,3,...) 


0 


: 4t 16 : 
= | tcos int dt = $ sin intt + cos intt 


r nt (nn) , 


0 (even n) 


-16/(nt) (odd n) 


= (cos mm 1) =| 
Tt) 
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(b) 


Then, by (7.26), the Fourier series expansion of F(f) is 
F(t) = 2-128(cos!nt++ cos 3nr++cosnt+...) 
= "2 COs 3 3 008 | 3 008 j ee 


or 


oy 1 


F(t) = 2--2 ¥ —— cos 1(2n - 1)at 
n=1 (2n- 1)’ 


Since F(t) = f(A) for 0 < t < 4, it follows that this Fourier series is representative 
of f(t) within this interval. Thus the half-range cosine series expansion of f(f) is 


f=t=2 veer cos (2n-1)nt (0<1<4) (7.30) 
Tt (2n-1) 


Half-range sine series. Define the periodic function G(f) by 


aw =| feist Were) 
f~f(-j)=t (-4<t<0) 


G(t + 8) = Gi) 


Then, since G(f) is an odd periodic function with period 8, it has a convergent 
Fourier series expansion given by (7.28). Taking t= 4 in (7.29), we have 


4 
b.= 2] osingonea (n=1,2,3,...) 


4 
= | t sin innt dt = ; Se cos intt + 16 sin intt 
2 4 
nt (n 


0 0 
= -2 cos nn = Say 
nt nth 
Thus, by (7.28), the Fourier series expansion of G(f) is 
G(t) = * (sin int isin int+tsin3nt-...) 


or 


G(t) = ‘yy cue sin = intt 


er 


Since G(t) = f(t) for 0 < t < 4, it follows that this Fourier series is repres- 
entative of f(¢) within this interval. Thus the half-range sine series expansion of 


S(O is 


co n+1 
ft) = 1= 8 SY sin tnt (0<t<4) (7.31) 
; noon 4 
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Graphs of the given function f(¢) and of the even and odd periodic expansions 
F(t) and G(A) are given in Figures 7.20(a), (b) and (c) respectively. 


Figure 7.20 f 


The functions f(4), 
F(t) and G(t) of 
Example 7.12. 


14 


15 


It is important to realize that the three different Fourier series representations 
(7.25), (7.30) and (7.31) are representative of the function f(t) = t only within the 
defined interval 0 < t < 4. Outside this interval the three Fourier series converge 
to the three different functions @(4), F(t) and G(9A), illustrated in Figures 7.17(b), 


7.20(b) and 7.20(c) respectively. 


7.3.3 Exercises 


Show that the half-range Fourier sine series 
expansion of the function f(A) = 1, valid for 
0<t<Z,is 


fg=ty mGn—bt (0<t<n) 
To 2n-1 


Sketch the graphs of both f(¢) and the periodic 
function represented by the series expansion 
for -3m <t < 3m. 


Determine the half-range cosine series expansion of 
the function f(t) = 2t—- 1, valid for 0 < ¢< 1. Sketch 
the graphs of both f(t) and the periodic function 
represented by the series expansion for 
2<t<2. 
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The function f(t) = 1 — ¢ is to be represented by 
a Fourier series expansion over the finite interval 
0 <t< 1. Obtain a suitable 


(a) full-range series expansion, 
(b) half-range sine series expansion, 
(c) half-range cosine series expansion. 


Draw graphs of f(¢) and of the periodic functions 
represented by each of the three series for 
-4<t<4, 


A function f(t) is defined by 
ft)=nt-t? 


and is to be represented by either a half-range 
Fourier sine series or a half-range Fourier cosine 


(0<t<n7m) 
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18 


Ig) 


series. Find both of these series and sketch the 
graphs of the functions represented by them for 
—2n<t< 2m. 


A tightly stretched flexible uniform string has its 
ends fixed at the points x = 0 and x =/. The midpoint 
of the string is displaced a distance a, as shown in 
Figure 7.21. If f(x) denotes the displaced profile of 
the string, express f(x) as a Fourier series expansion 
consisting only of sine terms. 


ff) 


a 


L a 
10) 1, l 

2 

Figure 7.21 Displaced string of Exercise 18. 


Repeat Exercise 18 for the case where the displaced 
profile of the string is as shown in Figure 7.22. 


f(x) 


1 
ql 


Figure 7.22 Displaced string of Exercise 19. 


7A 


20 


21 


Ze 


23 


A function /(#) is defined on 0 S ¢t S 7% by 


f=} Gees 3) 
0 (;2 Stn) 


Find a half-range Fourier series expansion 
of f(4) on this interval. Sketch a graph of 
the function represented by the series for 
—2n StS 20. 


A function /(‘) is defined on the interval 
-l<x</by 


f) = Fx =f) 


Obtain a Fourier series expansion of f(x) and sketch 
a graph of the function represented by the series for 
31 <x S31. 


The temperature distribution 7(x) at a distance x, 
measured from one end, along a bar of length 
L is given by 


T(x)=Kx(L—x) (O<x<L), K=constant 


Express 7(x) as a Fourier series expansion 
consisting of sine terms only. 


Find the Fourier series expansion of the function 
f(t) valid for -1 < t < 1, where 


tl <=12 420) 


no-| 
(0<t<1) 


cos Tt 


To what value does this series converge when 
t=1? 


Differentiation and integration of Fourier series 


It is inevitable that the desire to obtain the derivative or the integral of a Fourier series 
will arise in some applications. Since the smoothing effects of the integration pro- 
cess tend to eliminate discontinuities, whereas the process of differentiation has the 
opposite effect, it is not surprising that the integration of a Fourier series is more likely 
to be possible than its differentiation. We shall not pursue the theory in depth here; 
rather we shall state, without proof, two theorems concerned with the term-by-term 
integration and differentiation of Fourier series, and make some observations on their 


use. 
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7.4.1 


Theorem 7.3 


Example 7.13 
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Integration of a Fourier series 


A Fourier series expansion of a periodic function f(t) that satisfies Dirichlet’s con- 
ditions may be integrated term by term, and the integrated series converges to the 
integral of the function f(d). 


end of theorem 


According to this theorem, if f(t) satisfies Dirichlet’s conditions in the interval 
—% < ¢ <7 and has a Fourier series expansion 


FO = Fa) + » (a, cos nt + b, sin nt) 


n=1 


then for-ut St,< t= 


| noer=| jaar | (cos seein ti 
(pal ty 


t t 


1 1 


— |b (oar: : 
= Sao(t - ii) ar 2 E (cos nt, - cos nt) + rs (sin nt - sin nt,) 
n=l 


Because of the presence of the term } apt on the right-hand side, this is clearly not a 
Fourier series expansion of the integral on the left-hand side. However, the result can 
be rearranged to be a Fourier series expansion of the function 


at) = | S(t) dt — 3 aot 


t 


Example 7.13 serves to illustrate this process. Note also that the Fourier coefficients in 
the new Fourier series are —),/n and a,/n, so, from the observations made in Section 7.2.9, 
the integrated series converges faster than the original series for (4). If the given function 
f(t) is piecewise-continuous, rather than continuous, over the interval —m < ¢ < 7 then 
care must be taken to ensure that the integration process is carried out properly over the 
various subintervals. Again, Example 7.14 serves to illustrate this point. 


From Example 7.5, the Fourier series expansion of the function 


f= (a<t<n), f(t+2n)=f(n) 


is 


Integrating this result between the limits —z and ¢ gives 


t ry t 7 
| Par = | wars | (c1) cos ni a, 
—1 — n 


n=1 J -n 
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Example 7.14 


Solution 


that is, 
3 2 = (-1 i sin nt 
1 1 
3f = 40 t+4 > 2 (-t<t<7T) 
n=1 n 


Because of the term int on the right-hand side, this is clearly not a Fourier series 
expansion. However, rearranging, we have 


P-wt= 2y sin nt 
n=1 n 


and now the right-hand side may be taken to be the Fourier series expansion of the 
function 


ath=C-wt -n<t<n) 


ot + 2m) = g(t) 


Integrate term by term the Fourier series expansion obtained in Example 7.4 for the 
square wave 


go= 1 (-m <t <0) 
1 (0<t<n) 


S(t + 20) =f 


illustrated in Figure 7.7. 


From (7.21), the Fourier series expansion for f(f) is 


_ 4sin(2Qn-1)t 
Kj==- 
mT 2n-1 


We now need to integrate between the limits —m and ¢ and, owing to the discontinuity 
in f(é) at t= 0, we must consider separately values of ¢ in the intervals —m < ¢ < 0 and 
0<t<t. 


Case (i), interval —m < t < 0. Integrating (7.21) term by term, we have 
t t 
4 sin(2n - 1)t 
-l)dt=- dt 
ix ) >%| (2n- 1) 


n=1 


that is, 


(2n-1) 


—T 


~u4+m=-45 Feral 


(2n-1) 


n=1 


oo 1 | 
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7.4.2 


Theorem 7.4 
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It can be shown that 


ee Ly? 
Qn- an - 
(see Exercise 6), so that the above simplifies to 


>) enc (-n <t<0) (7.32) 


i 


Case (ii), interval 0 < t < 1m. Integrating (7.21) term by term, we have 


0 t . ; : 
(-1) dt+ 1dt=2y sin(2n- 1)t 4, 
a 0 Tt a (2n-1) 


n=1 
giving 


poin- ty soslen 9g <p<n) (7.33) 
n— (2n-1) 


n=1 


Taking (7.32) and (7.33) together, we find that the function 


-t (<t<0) 
f (0<1< 7) 


an=ii=| 


g(t + 27) = g(t) 


has a Fourier series expansion 


so =(etees conten as 
nm (2n-1) 


Differentiation of a Fourier series 


If f(A) is a periodic function that satisfies Dirichlet’s conditions then its derivative f’(1), 
wherever it exists, may be found by term-by-term differentiation of the Fourier series 
of f(t) if and only if the function f(t) is continuous everywhere and the function f’(#) has 
a Fourier series expansion (that is, f’(¢) satisfies Dirichlet’s conditions). 


end of theorem 


It follows from Theorem 7.4 that if the Fourier series expansion of f() is differenti- 
able term by term then f(t) must be periodic at the end points of a period (owing to the 
condition that f(f) must be continuous everywhere). Thus, for example, if we are deal- 
ing with a function /(f) of period 27 and defined in the range —m < t < 7m then we must 
have f(—1) = f(z). To illustrate this point, consider the Fourier series expansion of 
the function 
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Example 7.15 


Solution 


fO=t (W<t<n) 

S(t + 20) =fO 
which, from Example 7.7, is given by 

f(t) = 2(sint — }sin2r+ fsin3¢— fsin4t+...) 
Differentiating term by term, we have 

f(t) = 2(cost — cos 2t+ cos 3t— cos4t+...) 


If this differentiation process is valid then f(t) must be equal to unity for-n < t¢ <7. 
Clearly this is not the case, since the series on the right-hand side does not converge 
for any value of t. This follows since the nth term of the series is 2(—1)"*' cos nt and 
does not tend to zero as n > ~, 


If f(A is continuous everywhere and has a Fourier series expansion 


FO = Fa)+ x (a, cosnt + b, sin nt) 


n=1 


then, from Theorem 7.4, provided that f’(f) satisfies the required conditions, its Fourier 
series expansion is 


i Ox= Ss (nb, cos nt — na, sin nt) 


n=1 


In this case the Fourier coefficients of the derived expansion are nb, and na,, so, in 
contrast to the integrated series, the derived series will converge more slowly than the 
original series expansion for f(f). 


Consider the process of differentiating term by term the Fourier series expansion of the 
function 


f= (m<t<n),  f(t+2m=f() 
From Example 7.5, the Fourier series expansion of f(f) is 
f= in? +45 C1) cos nt (1 <t<n) 
n=1 n 


Since f(#) is continuous within and at the end points of the interval —t < ¢ < 1, we may 
apply Theorem 7.4 to obtain 


= (-1)"*' sin nt 
t=2 (Sh) ane —l<t<ut 
2 : ( ) 


which conforms with the Fourier series expansion obtained for the function 


foO=t (tW<t<n), f(t + 2m) =f 
in Example 7.7. 
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7.4.3 


Figure 7.23 Piecewise 
polynomial periodic 
function exhibiting 
jump discontinuities. 
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Coefficients in terms of jumps at discontinuities 


For periodic functions that, within a period, are piecewise polynomials and exhibit jump 
discontinuities, the Fourier coefficients may be determined in terms of the magnitude of 
the jumps and those of derived functions. This method is useful for determining describ- 
ing functions (see Section 7.8) for nonlinear characteristics in control engineering, 
where only the fundamental component of the Fourier series is important; this applies 
particularly to the case of multivalued nonlinearities. 

Consider a periodic function f(t), of period T, having within the time interval 
-}T <+t < }Ta finite number (m + 1) of jump discontinuities d), d,,..., dy 
at times f, tf, .. . t,, with t) = +7 and ¢,, = }7. Furthermore, within the interval 
ti.» <t<t,(s=1,2,...,m) let f(t) be represented by polynomial functions P,(f) 
(s=1,2,...,m), as illustrated in Figure 7.23. If f(A) is to be represented in terms of 
the Fourier series 


fO= + dy + By a, COS not + » b, sin not 


n=1 n=l 
then, from (7.4), 


m 


2 
D> 


t, 
| P(t) cos not dt 
t 


‘s-l1 


Defining the magnitude of the jump discontinuities as in Section 5.5.11, namely 


d, =f(t; + 0) ft; az 0) 


and noting that f) = —} 7 and ¢,, = } T, integration by parts and summation gives 


m t, 
g23b a sin not, + | P‘?(t) sin nora (7.34) 


s=l tei 


where P‘)(4) denotes the piecewise components of the derivative f‘(f) = f(A) in the 
generalized sense of (5.59). 
In a similar manner the integral terms of (7.34) may be expressed as 


f t 
m Ss . 1 m Ss 
»y | pe sin not dt = — > Je nor| P(t) cos nora 
n@ 


s=1 ty 
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where d‘” (s = 1, 2,..., m) denotes the magnitude of the jump discontinuities in the 
derivative f‘)(2). 
Continuing in this fashion, integrals involving higher derivatives may be obtained. 


However, since all P,(f) (s= 1, 2,..., m) are polynomials, a stage is reached when all 
the integrals vanish. If the degree of P,(f) is less than or equal to N fors=1,2,...,m 
then 
m N 
a, = = ba (-1)"'(na)*[d°” sin not, + (na) 'd?"*? cos not, 
nT Sot oo 


(n #0) (7.35) 


where d“ denotes the magnitudes of the jump discontinuities in the rth derivative of 
f(t) according to (5.59). 
Similarly, it may be shown that 


m N 
b= = 2 2 (-1)' (na) [d&” cos nat, — (nay 'd?"*” sin not] (7.36) 
and the coefficient a) is found by direct integration of the corresponding Euler formula 
2 T/2 
ay = 2 f(t) dt (7.37) 


-T/2 


Example 7.16 —_ Using (7.35)-(7.37), obtain the Fourier series expansion of the periodic function f(A) 
defined by 


nn=\' (a <t<0) 
; —2 (0<¢< 7) 


ft + 2m) =f) 


Solution In this case N = 2, and the graphs of f(t) together with those of its first two derivatives 
are shown in Figure 7.24. 


fOM 


fore 
ep? 


Figure 7.24 f(A), f(0), f() of Example 7.16. 
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Jump discontinuities occur at ¢ = —1, 0 and 7, so that m = 2. The piecewise poly- 
nomials involved and the corresponding jump discontinuities are 


(a) P=, P()=-2 
d, =-2, dy=nm°> +2 


(b) PY(H=2t PLY =0 
d®=0 dP=-2n 


() PPH=2, PHH=0 
d?=-2 d%=2 


with d\” = dS =0 forr > 2. 
Taking @= | (since T = 27) in (7.35) gives 


2 2 2 

: 1 1 : 

a, = 4 2 d, sin nt, - — by d‘? cos nt, + - Y d” sin nt, 
nt s=l NM sz) s=l 

Since ¢, = 0, t, = 7, sin 0 = sin nt = 0, cos 0 = 1 and cos na = (—1)", we have 


a,=2(-1)" (n=1,2,3,...) 
n 


Likewise, from (7.36), 


2 2 2 
«th — 1 (1): 1 (2) 
b= LS dcosmn 2S a sinnt,-) d; cos nt, 
nt s=l n s=l n s=1 


= Lf a24enteayen"- 4242-04] 
nt 7 
-1{(2-2)0-coeeen't Wa Tears) 
nM n 


and, from (7.37), 


0 T 
| Pars aay =!n’-2 
Tl — a 


Thus the Fourier expansion for f(f) is 


fit) = (t= 1) + Y 2-1)" cos nt 


n= 1 
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24 


25 


26 


7.4.4 Exercises 


Show that the periodic function 
fO=t 
S(t + 27) = f(t) 


has a Fourier series expansion 


Cre1<T) 


27 Tt ot , QU. 4.0. 3Tt 
t) = —| sin — — =sin — +-=sin — 
Ko al i ak ae ae 


- isin 42 +.) 
T 


By term-by-term integration of this series, show 
that the periodic function 


gar 
g(t + 2T) =a) 


has a Fourier series expansion 


Cr<r<T) 


2 
g(t) =1T° - AF {cos  - + ons 2 
T 


(Hint: A constant of integration must be introduced; 
it may be evaluated as the mean value over a period.) 


The periodic function 
hij=w’—-t? (a<t<n) 
A(t + 2m) = h(t) 


has a Fourier series expansion 


I(t) = 3x? +4( cos t- 4 cos 2t 
2 
+4 cos3t... } 
3 


By term-by-term differentiation of this series, 
confirm the series obtained for f(7) in Exercise 24 
for the case when T= 7. 


(a) Suppose that the derivative f’(t) of a periodic 
function f(t) of period 27 has a Fourier series 
expansion 


SO = $A + >) 4,.cos nt + pe sin nt 


n=1 n=1 


Show that 


2H) 


28 


(b) 


(c) 


Ay = + [frr) — f-W')] 


A, = (—1)"Ao + nb, 
B,=—na 


n n 


where ap, a, and b, are the Fourier coefficients 
of the function (2). 


In Example 7.6 we saw that the periodic function 
f=P+t (a<t<n) 
f+ 2m) =f) 

has a Fourier series expansion 


ft) = tn? + y 4 1)’ cos nt 


n= 1 


= y 7 (-1)" sin nt 
n 
n=1 
Differentiate this series term by term, and 
explain why it is not a Fourier expansion of the 
periodic function 


g(t) =2t+1 
g(t + 27) = g(t) 


Use the results of (a) to obtain the Fourier 
series expansion of g(t) and confirm your 
solution by direct evaluation of the coefficients 
using Euler’s formulae. 


(-—t<t< tT) 


Using (7.35)—(7.37), confirm the following Fourier 
series expansions: 


(a) 
(b) 


(c) 


(7.21) for the square wave of Example 7.4; 
the expansion obtained in Example 7.1 for the 
sawtooth wave; 

the expansion obtained for the piecewise- 
continuous function f(t) of Example 7.3. 


Consider the periodic function 


0 (-n <t<-—}n) 

mwm+2t (-!n<t<0) 
S(O = ; ; 

m-2t (O<t< 35m) 

0 Gn <t<n) 


S(t + 2m) =f) 


(a) Sketch a graph of the function for —4m < t < 4n. 
(b) Use (7.35)-(7.37) to obtain the Fourier series 
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‘ 1 
fit) =1n - 2° F(cos Lum 1) cos nt 2 (m<t<-jn) 
— ) f=) 2 Cin <t<3n) 
and write out the first 10 terms of this series. 


1 
(Note: Although the function /(f) itself has no a | 


jump discontinuities, the method may be used flt+2n)=f(t) 

since the derivative does have jump 

discontinuities. t 0<t<l 
(©) At= 


; b-¢ (S720) 
29 Use the method of Section 7.4.3 to obtain the 


Fourier series expansions for the following periodic ft+2)=f() 
functions: 
S+t (-~<1<0) 
(a) ft) = 0 (a"<t<0) (d) f(t)= 
f (0<t<n) 1-4 (O<t<} 


S(t + 2m) =f) ft+ D=fO 


yee a irdli\-alircae]) ieee frequency response and 
oscillating systems 


7.5.1 Response to periodic input 


In Section 5.7 we showed that the frequency response, defined as the steady-state 
response to a sinusoidal input A sin w?, of a stable linear system having a transfer func- 
tion G(s) is given by (5.101) as 


x, (0) = A|G(jo)| sin [wt + arg G(ja)] (7.38) 


By employing a Fourier series expansion, we can use this result to determine the 
steady-state response of a stable linear system to a non-sinusoidal periodic input. For a 
stable linear system having a transfer function G(s), let the input be a periodic function 
P(t) of period 27 (that is, one having frequency @ = n/T in rads“'). P(t) may be 
expressed in the form of the Fourier series expansion 


P(t)= Sag + by A, sin(no@t + @,) (7.39) 

n=1 
where A, and @, are defined as in Section 7.2.1. The steady-state response to each term 
in the series expansion (7.39) may be obtained using (7.38). Since the system is linear, 
the principle of superposition holds, so that the steady-state response to the periodic 
input P(t) may be obtained as the sum of the steady-state responses to the individual 
sinusoids comprising the sum in (7.39). Thus the steady-state response to the input P(f) is 


xa(0) = bayG(0) + ¥) 4,1 Gl jno)| sin [neat + 6, + arg G(jne)] (7.40) 


n=1 


www.2Ofile.org 


604 FOURIER SERIES 


Example 7.17 


Figure 7.25 (a) System 
and (b) input for 
Example 7.17. 


Solution 


There are two issues related to this steady-state response that are worthy of note. 


(a) For practical systems |G(j@)| > 0 as @ > ~, so that |G( jn@)| > 0 as n > in 
(7.40). As a consequence, the Fourier series representation of the steady-state 
response x,,(f) converges more rapidly than the Fourier series representation of 
the periodic input P(t). From a practical point of view, this is not surprising, since 
it is a consequence of the smoothing action of the system (that is, as indicated in 


Section 7.4, integration is a ‘smoothing’ operation). 


(b) There is a significant difference between the steady-state response (7.40) to a 
non-sinusoidal periodic input of frequency @ and the steady-state response (7.37) 
to a pure sinusoid at the same frequency. As indicated in (7.38), in the case of a 
sinusoidal input at frequency @ the steady-state response is also a sinusoid at the 
same frequency @. However, for a non-sinusoidal periodic input P(/) at frequency 
@ the steady-state response (7.40) is no longer at the same frequency; rather it 
comprises an infinite sum of sinusoids having frequencies n@ that are integer 
multiples of the input frequency @. This clearly has important practical implica- 
tions, particularly when considering the responses of oscillating or vibrating sys- 
tems. If the frequency nq@ of one of the harmonics in (7.40) is close to the natural 
oscillating frequency of an underdamped system then the phenomenon of reson- 


ance will arise. 


To someone unfamiliar with the theory, it may seem surprising that a practical 
system may resonate at a frequency much higher than that of the input. As indicated 
in Example 5.30, the phenomenon of resonance is important in practice, and it is there- 
fore important that engineers have some knowledge of the theory associated with 
Fourier series, so that the possible dominance of a system response by one of the higher 


harmonics, rather than the fundamental, may be properly interpreted. 


The mass—spring—damper system of Figure 7.25(a) is initially at rest in a position of 
equilibrium. Determine the steady-state response of the system when the mass is sub- 
jected to an externally applied periodic force P(t) having the form of the square wave 
shown in Figure 7.25(b). 


(b) 


From Newton’s law, the displacement x(f) of the mass at time f is given by 
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P(s) i xs) SO that the system may be represented by the block diagram of Figure 7.26. Thus the 
system transfer function is 


Figure 7.26 Block 0 (7.42) 


diagram for the system Ms’ +Bs+K 


of Figure 7.26. : : : : 
. From Example 7.4, the Fourier series expansion for the square wave P(t) is 


Pos 20 ines SS 4 sin St, ,Sin@n-|jt, 
Tt 


5 77s 2n-1 
that is, 
Pth=u()t+u()t+u,()t+...+u,O+... (7.43) 
where 
u,(t) = 40 s so smn Ut (7.44) 


2n-1 


Substituting the given values for M, B and K, the transfer function (7.42) becomes 


1 
G(s) = —————_ 
10s° + 0.55 + 250 
Thus 
1 250-10@  .0.5@ 
G(jo) = — = et ji 
-10@ +0.5j@ + 250 D D 


where D = (250 — 10@°)* + 0.25@’, so that 


|GGo)| = | - 1007)" + easel 


D 
Se ees (7.45) 
JD \[(250 - 10a@7)? + 0.250 ] 
arg G(jo) = —tan™ (82) (7.46) 
250 - 10a 


Using (7.38), the steady-state response of the system to the mth harmonic u,(t) given by 
(7.44) is 


40 
m(2n - 1) 


where |G(jq@)| and arg G(j@) are given by (7.45) and (7.46) respectively. The steady- 
state response x,,(¢) of the system to the square-wave input P(f) is then determined as 
the sum of the steady-state responses due to the individual harmonics in (7.43); that is, 


Xesn(£) = |G(j(2n — 1))| sin[(2n — 1)t + arg G(j(2n — 1))] (7.47) 


x= YX (7.48) 


n=1 


where x,,,(¢) is given by (7.47). 
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Figure 7.27 
Steady-state response 
of system of 
Figure 7.25. 


Evaluating the first few terms of the response (7.48), we have 


er tan" 95) 
T™ /[(250 - 10)? + 0.25] 240 


= 0.053 sin(t - 0.003) 


Xwol(ty= ee sin |3t- tanr'( 15) 
3% [(250 - 90)? + 2.25] 160 


= 0.027 sin(3t - 0.009) 


40 1 . gee 
33(t) = — ——— t-t —= 
Xo53(1) 5m (6.25) sin [ an 0 | 


= 1.02 sin(St- $n) 


Xesa(t) = pa ——__1____ sin| 7t- tan'( 35) 
7M [(250 - 490) + 12.25] = 


= 0.0076 sin(7t - 3.127) 
Thus a good approximation to the steady-state response (7.48) is 
x,t) = 0.053 sin(t — 0.003) + 0.027 sin(3t — 0.54) + 1.02 sin(St — $1) 
+ 0.0076 sin(7t — 3.127) (7.49) 


The graph of this displacement is shown in Figure 7.27, and it appears from this that 
the response has a frequency about five times that of the input. This is because the term 
1.02 sin(St — $m) dominates in the response (7.49); this is a consequence of the fact that 
the natural frequency of oscillation of the system is \(K/M) = Srads"', so that it is in 
resonance with this particular harmonic. 

In conclusion, it should be noted that it was not essential to introduce transfer func- 
tions to solve this problem. Alternatively, by determining the particular integral of the 
differential equation (7.41), the steady-state response to an input A sin wt is determined as 


= A sin(@t- Q) _ __@B 
X,5(t) = ee ee MS 
\L(K - Ma) + Boo | K- Mo 
giving x,,,(¢) as in (7.48). The solution then proceeds as before. 
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7.5 ENGINEERING APPLICATION: FREQUENCY RESPONSE AND OSCILLATING SYSTEMS 


7.5.2 Exercises 


Determine the steady-state current in the circuit of 
Figure 7.28(a) as a result of the applied periodic 
voltage shown in Figure 7.28(b). 


R= 3002 


Ee 
C0) fo 
C=4~x10-6F 


L=0.02H 
{a) 


Figure 7.28 (a) Circuit of Exercise 30; 
(b) applied voltage. 


Determine the steady-state response of the mass— 
spring—damper system of Figure 7.29(a) when the 
mass is subjected to the externally applied periodic 
force f(t) shown in Figure 7.29(b). 

What frequency dominates the response, and 
why? 


B=0.5 kgs“! 


fro 


(b) 


Figure 7.29 (a) Mass—spring—damper system of 
Exercise 31; (b) applied force. 
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Determine the steady-state motion of the mass of 
Figure 7.30(a) when it is subjected to the externally 
applied force of Figure 7.30(b). 


LZ 
K=80Nm! 


fo M =20kg 


(a) 


fo 


(b) 


Figure 7.30 (a) Mass—spring—damper system of 
Exercise 32; (b) applied force. 


Determine the steady-state current in the circuit 
shown in Figure 7.31(a) when the applied voltage is 
of the form shown in Figure 7.31(b). 


-§ 
icon 10°F 
(ea — 7 
0.4H 

(a) 
e(f) 
100 
.¢) 

0.02 0.04 0.06 ¢ 


(b) 


Figure 7.31 (a) Circuit of Exercise 33; (b) applied 
voltage. 
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7.6.1 


Complex form of Fourier series 


An alternative to the trigonometric form of the Fourier series considered so far is the 
complex or exponential form. As a result of the properties of the exponential function, 
this form is easily manipulated mathematically. It is widely used by engineers in prac- 
tice, particularly in work involving signal analysis, and provides a smoother transition 
from the consideration of Fourier series for dealing with periodic signals to the con- 
sideration of Fourier transforms for dealing with aperiodic signals, which will be dealt 
with in Chapter 8. 


Complex representation 


To develop the complex form of the Fourier series 


SO= a) + > a, COS n@t + > b, sinnot (7.50) 


n=l n=1 


representing a periodic function f(t) of period 7, we proceed as follows. Substituting 
the results 


sin not = °F (ein = enor) 


— 1lf,jnot —jnot 
cos nat = 5(e"" + eM) 


into (7.50) gives 


jnot —jnot co jnot —jnot 
+e Sc 


fit) = bay ») aj + 


n=1 


n 


= tay i. »y ta (ee fe ere) ze x 7 1jb,(e" = ev) 
n= 


n=1 


= Sag + Y [5(ay ~ ibn) + Hay + jy) OP] (7.51) 
Writing 
Co= + ap, Cn = (a, ~ j>,), Cy = a = (a, + 1) (7.52) 


(7.51) becomes 


f=cyt+ > c, em > c_,e 
n=1 


n=1 


= jnot jnot 
=o+ > ce +) ce 
n=1 


n=-1 


= jnaot : 0 
- > ce", since cye°= cy 


i 
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Thus the Fourier series (7.50) becomes simply 


co 


fo= ¥ cer (7.53) 


n=—0co 


which is referred to as the complex or exponential form of the Fourier series expan- 
sion of the function f(f). 

In order that we can apply this result directly, it is necessary to obtain a formula for 
calculating the complex coefficients c,. To do this, we incorporate the Euler formulae 
(7.4) and (7.5) into the definitions given in (7.52), leading to 


Cy = by = ; | fit) dt (7.54) 


d 


d+T 


d d 


Cy = $(4, ~ jn) = || f(t) cos nat dt -i| f(t) sin nora 


d+T 
=. | f()(cos nwt — j sin nat) dt 


d 
d+T 

=F | fayem"at (7.58) 
d 


d+T 


C= S(a, + jb,) = 1] f(A)(cos nwt + j sin not) dt 


d 


d+T 
-1 | fit) edt (7.56) 
- d 
From (7.54)-(7.56), it is readily seen that for all values of 
d+T 
go | few dt (7.57) 
r d 
Summary 


In summary, the complex form of the Fourier series expansion of a periodic function 
f(t), of period T, is 


fo= ¥ 0" (7.53) 
where 
d+T 
ee ; | fer'dt (n=0,+1,42,...) (7.57) 
d 
In general the coefficients c, (n = 0, 1, +2, ...) are complex, and may be expressed 
in the form 
Ch = Ic,| etn 
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Example 7.18 


Figure 7.32 Function 
f(t) of Example 7.18. 


Solution 


where |c,|, the magnitude of c,, is given from the definitions (7.52) by 


le,| = yiGa,)? + Gb,)7] = 5 \(a? + 6?) 


so that 2|c,,| is the amplitude of the mth harmonic. The argument @, of c,, is related to the 
phase of the nth harmonic. 


Find the complex form of the Fourier series expansion of the periodic function f(‘) 
defined by 


f()=cos}t (-t<t<m, f(tt+2n=f(h 


iG) 


-3n —1 Oo T 3n 


A graph of the function f(t) over the interval —3m < ¢ < 3m is shown in Figure 7.32. 
Here the period 7 is 27, so from (7.57) the complex coefficients c,, are given by 


Tu Tq 
C,= | cole dro) (eee di 
2n) z 4n} 


TT 

1 =j(n-1/2 -j(n+1/2 

= re (e j(n wie j(n+ * dt 
—1 


1 —2 eIGn D2 2 @ ientiee ad 
~An| j(Qn-1) j(2n+1) 


—T 


j ew" eit? ev ein? el™ in? el" elt? 
= st YH 
vl eee 2n+1 Gee Tr) 


Now e”™ = cos $n + jsin$™=j, eo" =-j and e”* = e"" = cosnn = (-1)’, so that 


a= ti tet - te” 
2m\2n-1 2n+1 2n-1 2n+1 
=i | )- =2(-1)" 

tm \2n+1 2n-1 (4n’ - 1)n 


Note that in this case c, is real, which is as expected, since the function f() is an even 
function of tf. 
From (7.53), the complex Fourier series expansion for f(f) is 


7 co 2(-1)"" fai 
He) Dy (4n’ - Dn 


This may readily be converted back to the trigonometric form, since, from the defini- 
tions (7.52), 
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ay = 2Co, a, = Cy + Cs b, = JCC, ~ c%) 
so that in this particular case 
n+1 n+l 
ee a, =2 202) |= Sea b,=0 
ut nm 4n°+1 m4n°-1 


Thus the trigonometric form of the Fourier series is 


fit)= +8 SO os nt 


Tt “7 4n -1 


which corresponds to the solution to Exercise I(e). 


Example 7.19 = Obtain the complex form of the Fourier series of the sawtooth function f(f) defined by 


f= O<1< 27),  fll+27)=f0 


Figure 7.33 Function f(4) 
of Example 7.19. 


Solution A graph of the function f(t) over the interval -6T < t < 67 is shown in Figure 7.33. 
Here the period is 27, that is w = n/T, so from (7.57) the complex coefficients c,, are 


given by 
| 2f a a) 
-~t t elt dea <t inn ye 
o 4] 40 ar} rt? 
acre en a 
= = t euinlr = : euinlr (n # 0) 
T |\-jnt (jnt)” ; 
Now e?”" =e? = 1, so 


1 2 2 2 2 
n= ee 24 (n #0) 
a [-jnu (nt) (nt) 


In the particular case n = 0 


2T 27 
| noar=+| tar= 7, [i] =2 
OP |. Fle fF ly 


Thus from (7.53) the complex form of the Fourier series expansion of f(f) is 


a | = 1 i es j 7 7 
fit) =2+ »y us em 24 y = 


n=—0o n=1 n=—0o 
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Noting that j = e!”°, this result may also be written in the form 


2ra1, 
f= 2 += - ai(nnt/T+1/2) 
f) y 7e 


As in Example 7.18, the Euler coefficients in the corresponding trigonometric series are 


Ay = 2c) = 4, a, = C, + cX=0, b= ile + ot) = i+ 24) = -4 
ngtu AT nt 


so that the corresponding trigonometric Fourier series expansion of f(f) is 


4c1. nut 
jee -sle 
S(t) yr - 


n=1 


which corresponds to the solution of Example 7.11 when T= 2. 


7.6.2 The multiplication theorem and Parseval’s theorem 


Two useful results, particularly in the application of Fourier series to signal analysis, 
are the multiplication theorem and Parseval’s theorem. The multiplication theorem 
enables us to write down the mean value of the product of two periodic functions over 
a period in terms of the coefficients of their Fourier series expansions, while Parseval’s 
theorem enables us to write down the mean square value of a periodic function, which, 
as we will see in Section 7.6.4, determines the power spectrum of the function. 


Theorem 7.5 The multiplication theorem 


If f(é and g(t) are two periodic functions having the same period 7 then 


: | fag() a= ¥ c,d? (7.58) 


Cc n=—co 


where the c,, and d,, are the coefficients in the complex Fourier series expansions of 
f(é and g(t) respectively. 


Proof Let f(#) and g(t) have complex Fourier series given by 


co 


f)=>¥ oe" (7.59a) 


with 


c+T 
ee ; | few" dT (7.59b) 
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and 
g(t) = by d, einmiT 
with 
1 c+T 
d=- t e J2nlT qt 
; aN g(t) 
Then 


1] fdg() dt = a » é om hee 


c+T 
— 1 jn2nt/T 
= | t dt 
ae e [ewe | 


Since d_, = d* the complex conjugate of d,, this reduces 


n> n> 


e+T a 
; | | fg ar= end 


Cc n=—co 


(7.60a) 


(7.60b) 


using (7.59a) 


assuming term-by-term 
integration is possible 
using (7.59b) 


to the required result: 


end of theorem 


In terms of the real coefficients a,, b, and @,, 8, of the corresponding trigonometric 


nn 


Fourier series expansions of f(t) and g(f), 


At) = tay + ey ay cos{ “2 + 2 b, sin( "22 


n=1 


g(t) = 40t)+ »y a, cos{ “2A + Y B, sin( 22 


n=1 n=1 


and using the definitions (7.52), the multiplication theorem result (7.58) reduces to 


c+T 
1 S(O)gO dt = > ca, + Cody + » a 
e n=1 n=1 


=! agg + 2S [(a,— 58, Xe, + jB,) + (a, + i0,)(0%, — 58,91 


n=1 


giving 


(Arie 
1 SOs dt = Fly ar a (4,06, ap b,B,) 
jo n=1 


www.2Ofile.org 


614 FOURIER SERIES 


Theorem 7.6 Parseval’s theorem 


If f( is a periodic function with period 7 then 


; | [OPdi= y' c.ct= y Ie.P (7.61) 


c n=-0o n=-co 


where the c, are the coefficients in the complex Fourier series expansion of f(¢). 


Proof This result follows from the multiplication theorem, since, taking g(t) = f(t) in (7.58), 
we obtain 


1 [fordr= ¥ cct= > le? 


n=—90 n=—co 


end of theorem 


Using (7.60), Parseval’s theorem may be written in terms of the real coefficients a,, 
and b,, of the trigonometric Fourier series expansion of the function f(f) as 


; | [foPar= ba +19 (a + 8%) (7.62) 


@ n=l 


The root mean square (RMS) value faus of a periodic function f(t) of period T, defined 
by 


Fins= 5 | _ Lope 


may therefore be expressed in terms of the Fourier coefficients using (7.61) or (7.62). 


Example 7.20 __ By applying Parseval’s theorem to the function 


2t 


fO=G Ostst, Mee2na1) 


considered in Example 7.19, show that 


Solution From Example 7.19, the coefficients of the complex Fourier series expansion of f(t) are 
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Thus, applying the Parseval’s theorem result (7.61), noting that the period in this case 
is 27, we obtain 


an -1 oo 
Z| [OP d= 5+ leh + ¥ lek 
n=1 


0 n=—00 


giving 


1 [42 72) 
| 4 ar=4425 (4) 
27), F nn 


n=1 


which reduces to 


Discrete frequency spectra 


In expressing a periodic function /(/) by its Fourier series expansion, we are decompos- 
ing the function into its harmonic or frequency components. We have seen that if f(4) 
is of period 7 then it has frequency components at frequencies 

_2nm _ _ 

O, = <= NW (n=1,2,3,...) (7.63) 
where @, is the frequency of the parent function f(t). (All frequencies here are meas- 
ured in rads.) 

A Fourier series may therefore be interpreted as constituting a frequency spectrum 
of the periodic function f(t), and provides an alternative representation of the function 
to its time-domain waveform. This frequency spectrum is often displayed by plotting 
graphs of both the amplitudes and phases of the various harmonic components against 
angular frequency @,. A plot of amplitude against angular frequency is called the 
amplitude spectrum, while that of phase against angular frequency is called the phase 
spectrum. For a periodic function (1), of period T, harmonic components only occur 
at discrete frequencies ,, given by (7.59), so that these spectra are referred to as dis- 
crete frequency spectra or line spectra. In Chapter 8 Fourier transforms will be used 
to define continuous spectra for aperiodic functions. With the growing ability to process 
signals digitally, the representation of signals by their corresponding spectra is an 
approach widely used in almost all branches of engineering, especially electrical engin- 
eering, when considering topics such as filtering and modulation. An example of the 
use of a discrete spectral representation of a periodic function is in distortion measure- 
ments on amplifiers, where the harmonic content of the output, measured digitally, to a 
sinusoidal input provides a measure of the distortion. 
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Figure 7.34 Real 
discrete frequency 
spectrum. 


If the Fourier series expansion of a periodic function f(t), with period 7, has been 
obtained in the trigonometric form 


f(t) = fay t 2 a, COS (2424) + 2 b, sin (2224) 


then, as indicated in Section 7.2.2, this may be expressed in terms of the various har- 
monic components as 


fit) = A+ ¥ A, sin (2am + 6,) (7.64) 


n=l 
where 


1 Wired 2 
Ay = 540, A, = \(a n v b;) 


2 


and the @, are determined by 


: b, a, 
sin @, = 4.’ cos @,, = ya 
In this case a plot of A, against angular frequency @, will constitute the amplitude 
spectrum and that of @, against @, the phase spectrum. These may be incorporated in 
the same graph by indicating the various phases on the amplitude spectrum as illus- 
trated in Figure 7.34. It can be seen that the amplitude spectrum consists of a series 
of equally spaced vertical lines whose lengths are proportional to the amplitudes of the 
various harmonic components making up the function f(‘). Clearly the trigonometric 
form of the Fourier series does not in general lend itself to the plotting of the discrete 
frequency spectrum, and the amplitudes 4, and phases @, must first be determined from 
the values of a, and b, previously determined. 


Amplitude A,, 


Wy 2W9 3u9 49 S5wo 


Frequency w,, 


In work on signal analysis it is much more common to use the complex form of the 
Fourier series. For a periodic function f(t), of period 7, this is given by (7.53), with the 
complex coefficients being given by 


c, =|c, |e" (n=0, +1, +2,...) 


in which |c,| and @, denote the magnitude and argument of c, respectively. Since in 
general c, is a complex quantity, we need two line spectra to determine the discrete 
frequency spectrum; the amplitude spectrum being a plot of |c,| against @, and the 
phase spectrum that of @, against @,. In cases where c, is real a single spectrum may be 
used to represent the function /(#). Since |c_,| = |c*| =|c,|, the amplitude spectrum will 
be symmetrical about the vertical axis, as illustrated in Figure 7.35. 
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form of the amplitude 
spectrum. 


Example 7.21 


Solution 
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— 609 —Swg—49-3W9- 29 —H9 


Wo 209 3wWo 4g Sig 6wg 
Frequency w,, 


Note that in the complex form of the discrete frequency spectrum we have com- 
ponents at the discrete frequencies 0, +@), +2@ , +3@p, ... ; that is, both positive and 
negative discrete frequencies are involved. Clearly signals having negative frequencies 
are not physically realizable, and have been introduced for mathematical convenience. 
At frequency n@, we have the component e!"”»’, which in itself is not a physical signal; 
to obtain a physical signal, we must consider this alongside the corresponding com- 
ponent e7”"°' at the frequency —n@,, since then we have 


elo! 4 "0! — 2 cosnWol (7.65) 


Plot the discrete amplitude and phase spectra for the periodic function 


f=% O<1< 27),  fe+27)=/W) 


of Example 7.19. Consider both complex and real forms. 


In Example 7.19 the complex coefficients were determined as 
i a, a ee ae 
nth 
Thus 


eis 2/nm (n=1,2,3,...) 
" |-2/nn (n =-1,-2,-3,...) 


ts: Gee OF cs) 


mats =| ; 
—;n (n=—1,-2,-3,...) 


The corresponding amplitude and phase spectra are shown in Figures 7.36(a) and (b) 
respectively. 

In Example 7.19 we saw that the coefficients in the trigonometric form of the Fourier 
series expansion of /(f) are 
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Aly als ow 


~4mg- 309 -20 ~Wo 


Wg 2009 3wWo 4wo wo 209 3wWo 49 


Frequency @, Frequency w, 


(b) 


Figure 7.36 Complex discrete frequency spectra for Example 7.21, with @, = 1/T: (a) amplitude spectrum; 


(b) phase spectrum. 


Figure 7.37 Real 
discrete frequency 
spectrum for 
Example 7.21 
(corresponding to 


sinusoidal expansion). 


Amplitude A,, 


wo 20 3a 4a Sao 6wWo 


Frequency w,, 


so that the amplitude coefficients in (7.63) are 
Ay=2, A,=— (n=1,2,3,...) 
nt 


leading to the real discrete frequency spectrum of Figure 7.37. 


Since |c,| = $\(a, + b;) = $A,, the amplitude spectrum lines in the complex form 
(Figure 7.36) are, as expected, halved in amplitude relative to those in the real repre- 
sentation (Figure 7.37), the other half-value being allocated to the corresponding 
negative frequency. In the complex representation the phases at negative frequencies 
(Figure 7.36b) are the negatives of those at the corresponding positive frequencies. In 
our particular representation (7.64) of the real form the phases at positive frequencies 
differ by } between the real and complex form. Again this is not surprising, since from 
(7.65) we see that combining positive and negative frequencies in the complex form 
leads to a cosinusoid at that frequency rather than a sinusoid. In order to maintain equal- 
ity of the phases at positive frequencies between the complex and real representations, 
a cosinusoidal expansion 


ft) = Ay+ A, c0s (= + 6,) (7.66) 


n=1 
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Figure 7.38 Real 
discrete frequency 
spectrum for 
Example 7.21 
(corresponding 

to cosinusoidal 
expansion). 


Example 7.22 


Figure 7.39 
Infinite train of 
rectangular pulses 
of Example 7.22. 


Solution 
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Amplitude A,, 


Wp 209 3wW9 49 Sag 


Frequency w,, 


of the real Fourier series is frequently adopted as an alternative to the sinusoidal series 
expansion (7.64). Taking (7.66), the amplitude spectrum will remain the same as for 
(7.68), but the phase spectrum will be determined by 


. b ay, 
sing, =-—, cos¢,=— 
%, A A 


n n 


showing a phase shift of +1 from that of (7.64). Adopting the real representation (7.66), 
the corresponding real discrete frequency spectrum for the function f(‘) of Example 
7.21 is as illustrated in Figure 7.38. 


Determine the complex form of the Fourier series expansion of the periodic (period 27) 
infinite train of identical rectangular pulses of magnitude A and duration 2d illustrated 
in Figure 7.39. Draw the discrete frequency spectrum in the particular case when d= + 
and T= }. 


2d 2d 2d 2d 2d 
Ke— 1 —— — 1 eI eI 
4T 3T oT pr 40d ¢ aT 37 4T t 


Over one period —7 < t < T the function f(‘) representing the train is expressed as 
0 (-T<t<-d) 
Kth=4A (-d<t<d) 
0 (d<t<T) 
From (7.57), the complex coefficients c, are given by 
C, = 1 fit) emir dt e a5 A etl at = A <r aon (n x 0) 
2T | _. ar} 2T | jnt .f 


jand/T —jntd/T 7 
~Aig -e = Asin ( Wad | — Ad sintand/t) Bd eps. 
nt j2 nt T  ntd/T 
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In the particular case when n = 0 


T d 
1 1 Ad 
=— t)dt=— Adt=—= 
Co an vil ) an 7 
so that 
c, = sine ( #24) (n=0, +1, #2,...) 
T T 


where the sine function is defined by 


sin ¢ 
—— (t#0 
sinc tf = t ( ) 
1 (t=0) 


Thus from (7.53) the complex Fourier series expansion for the infinite train of pulses 


S(O 1s 


Figure 7.40 Discrete 
amplitude spectrum 
for an infinite train of 
pulses when d= « and 
T=3: 


2 


Kd = > “4 sine ( #24) oe 


As expected, since /(f) is an even function, c,, is real, so we need only plot the discrete 
amplitude spectrum to represent f(f). Since the amplitude spectrum is a plot of |c,| 
against frequency n@ >, with @, =7/T, it will only take values at the discrete frequency 
values 


In the particular case d= i, 


frequency values 


T= i @) = 2m the amplitude spectrum will only exist at 


0, +27, +4n,... 
Since in this case 
¢,= tAsineinn (n=0,+1,+2,...) 


noting that sinc } 7m = 0 when } nm = mm or n = 5m (m= 41, +2, .. . ), the spectrum is 
as shown in Figure 7.40. 


Amplitude ic,! 


-l5w9 — - 10a —Su a9 Wo Swo 109 1S5wg Frequency wy 
-30n —20n -10n  -2n 2n 107 200 30n 
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Figure 7.41 
Graph of sinc ¢. 


7.6.4 


7.6 COMPLEX FORM OF FOURIER SERIES 621 


As we will see in Chapter 8, the sinc function sinc ¢ = (sin #)/t plays an important role 
in signal analysis, and it is sometimes referred to as the sampling function. A graph of 
sinc ¢ is shown in Figure 7.41, and it is clear that the function oscillates over intervals 
of length 27 and decreases in amplitude with increasing ¢. Note also that the function 
has zeros att=t+nt (n= 1,2,3,...). 


Power spectrum 


The average power P associated with a periodic signal f(4), of period T, is defined as 
the mean square value; that is, 


p=1 | LAOP dt (7.67) 


d 


For example, if f(t) represents a voltage waveform applied to a resistor then P represents 
the average power, measured in watts, dissipated by a | Q resistor. 
By Parseval’s theorem (Theorem 7.6), 


P= jaytz> (a, +5,) (7.68) 


n=1 


Since 


the power in the nth harmonic is 


P=1(a2 +b? (7.69) 


n n 


and it follows from (7.68) that the power of the periodic function f(¢) is the sum of the 
power of the individual harmonic components contained in f(t). 
In terms of the complex Fourier coefficients, Parseval’s theorem gives 


PS > jar (7.70) 


n=—co 


As discussed in Section 7.6.3, the component e!”o" at frequency @, = n@ , Wy) = 2n/T, 
must be considered alongside the component e~”«' at the corresponding negative fre- 
quency —q,, in order to form the actual mth harmonic component of the function /(?). 
Since |c_,|’ =|c*|? =|c,|’, it follows that the power associated with the nth harmonic is 
the sum of the power associated with e!”°" and e?”"; that is, 


www.2Ofile.org 


622 FOURIER SERIES 


Example 7.23 


Solution 


P,, = 2\c,/ (7.71) 


which, since |c,,| = 3 (a? + b?), corresponds to (7.69). Thus in the complex form half 
the power of the nth harmonic is associated with the positive frequency and half with 
the negative frequency. 

Since the total power of a periodic signal is the sum of the power associated with 
each of the harmonics of which the signal is composed, it is again useful to consider a 
spectral representation, and a plot of |c,|’ against angular frequency @, is called the 
power spectrum of the function f(t). Clearly such a spectrum is readily deduced from 
the discrete amplitude spectrum of |c,,| against angular frequency @,,. 


For the spectrum of the infinite train of rectangular pulses shown in Figure 7.39, deter- 
mine the percentage of the total power contained within the frequency band up to the 
first zero value (called the zero crossing of the spectrum) at 10mrads™’. 


From (7.67), the total power associated with the infinite train of rectangular pulses f(f) is 


T d 
xxl tae l | 
pt] 70) dt a dt 


which in the particular case when d= 4 and T= } becomes 
1/10 
P= | A dt= tA’ 
-1/10 
The power contained in the frequency band up to the first zero crossing at 10mrads"' is 
P= +2Uci +++) 
where 
c, = +A sine int 
That is, 
P, = 4A’ + 2A°(sine’ im + sinc’ 2m + sinc’ 2m + sinc’ ) 
= + A’[1 + 2(0.875 + 0.756 + 0.255 + 0.055)] = ; A’(0.976) 


Thus P, = 0.976P, so that approximately 97.6% of the total power associated with (4) 
is contained in the frequency band up to the first zero crossing at 10mrads"'. 


Suppose that a periodic voltage v(t), of period 7, applied to a linear circuit, results 
in a corresponding current i(f), having the same period 7. Then, given the Fourier series 
representation of both the voltage and current at a pair of terminals, we can use the 
multiplication theorem (Theorem 7.5) to obtain an expression for the average power P 
at the terminals. Thus, given 


v(t) = x Cc el2nniT i(t) = x d e2nmiT 
n > n 


n=—co n=—0o 
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the instantaneous power at the terminals is vi and the average power is 


n=—c0 


1 d+T si 
P=- i dt = 
ay . 2 


or, in terms of the corresponding trigonometric Fourier series coefficients a, b,, and 


a, > Pn 


P = £ & Bo + :> (a,,Q,, + b,B,) 
n=1 


7.6.5 Exercises 


34 — Show that the complex form of the Fourier series 
expansion of the periodic function 


f= (am<t<n) 
f(t + 2m) =f 
is 


f)=E+y 3c 


n=0 n 


Using (7.52), obtain the corresponding 
trigonometric series and check with the 
series obtained in Example 7.5. 


35 Obtain the complex form of the Fourier series 
expansion of the square wave 
0 C2<1<0) 


no={" (2122) 
f(t+4) =f) 


Using (7.52), obtain the corresponding 
trigonometric series and check with the 
series obtained in Example 7.7. 


36 Obtain the complex form of the Fourier 
series expansion of the following periodic 
functions. 


(-1<t<0) 
(0<t<m) 


(a) f(t) = | 
f(t + 2m) = ft) 


asinat (0<1<iT) 
(b) f(t) = 
GT <t<T) 


ft+T)=f/, T=2n/o 


© fo -? (-n <t <0) 
1 (<t<n) 
f(t + 2m) =f) 
(d) ff =|sint| (t<t< 7) 


Sit + 2m) =f 


37 A periodic function f(t), of period 21, is defined 
within the period —t < t < 7% by 

0 (-n<t<0) 

1 (<t<n) 


Sit) -| 


Using the Fourier coefficients of f(t), together with 
Parseval’s theorem, show that 


yee 
S(2n-1y * 


(Note: The Fourier coefficients may be deduced 
from Example 7.7 or Exercise 35.) 


38 (a) Show that the Fourier series expansion of the 
periodic function 


f() =500nt (0<1t< <) 
fit + 5) =fO 


may be expressed as 


fit)=5n- 10>. 1 cin 100nnt 
n 


n=l 


(b) Using (7.62), estimate the RMS value of f(7) by 
(i) using the first four terms of the Fourier 
series; 
(ii) using the first eight terms of the Fourier 
series. 
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(c) Obtain the true RMS value of f(t), and hence (a) Obtain expressions for the coefficients c,, of the 
determine the percentage errors in the complex Fourier series representation of v(f), 
estimated values obtained in (b). and write down the values of the first five 

non-zero terms. 
39 A periodic voltage v(#) (in V) of period 5 ms and (b) Calculate the power associated with each of 
specified by the first five non-zero terms of the Fourier 
expansion. 
v(t) = fe (0 <¢< 1.25 ms) (c) Calculate the total power delivered to the 
QO (1.25 ms < ¢ < 5 ms) 15 Q resistor. 


(d) What is the percentage of the total power 
delivered to the resistor by the first five 
is applied across the terminals of a 15 Q resistor. non-zero terms of the Fourier series? 


v(t + 5 ms) = v(t) 


Orthogonal functions 


As was noted in Section 7.2.2, the fact that the set of functions {1, cos wf, sin at, 
..., cosnat, sinnot,... } is an orthogonal set of functions on the intervald <t<d 
+ T was crucial in the evaluation of the coefficients in the Fourier series expansion of a 
function f(f). It is natural to ask whether it is possible to express the function f(f) as a 
series expansion in other sets of functions. In the case of periodic functions /(f) there 
is no natural alternative, but if we are concerned with representing a function f(t) only 
in a finite interval ¢; < ¢ S ¢, then a variety of other possibilities exist. These possibil- 
ities are drawn from a class of functions called orthogonal functions, of which the 
trigonometric set {1, cos wf, sin @t,..., cosnat, sinna@t} is a particular example. 


7.7.1 Definitions 


Two real functions /(f) and g(f) that are piecewise-continuous in the interval ¢,; <¢<¢, 
are said to be orthogonal in this interval if 


| “fidg(t) dt = 0 


a 


A set of real functions @,(f), @,(0), . . . = {0,(f)}, each of which is piecewise-continuous 
on ¢; < f S tb, is said to be an orthogonal set on this interval if @,(¢) and @,,(f) are 
orthogonal for each pair of distinct indices n, m; that is, if 


| b,OOnAt) dt=0 (n#m) (7.72) 


ty 


We shall also assume that no member of the set {@,(f)} is identically zero except at a 
finite number of points, so that 


| eou=n (m= 1, 2,3,...) (7.73) 


ty 


where y,, (m= 1, 2,...) are all non-zero constants. 
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Example 7.24 


Example 7.25 


Solution 


An orthogonal set {@,(f)} is said to be orthonormal if each of its components is also 
normalized; that is, y,, = 1 (m= 1, 2, 3,...). We note that any orthogonal set {@,(2)} 
can be converted into an orthonormal set by dividing each member 9@,,(t) of the set by \y,,. 


Since (7.6)—(7.10) hold, 
{1, cost, sint, cos 2¢, sin2t,..., cosnt, sinnt} 


is an orthogonal set on the interval d < t < d+ 2n, while the set 


sig 


1 cost sint cosnt sinnt 
1 x f > 1 2. 1 > A 
\(2m) \n Vn \u VT 


forms an orthonormal set on the same interval. 
The latter follows since 


d+21 2 
| u dt=1 
d+2n 2 d+2n , , 2 
(cos) es (saz!) @=1 GH=t95.. 
\t 7 yt 


d 


The definition of orthogonality considered so far applies to real functions, and has 
to be amended somewhat if members of the set {@,(t)} are complex functions of the real 
variable ¢. In such a case the set {@,(t)} is said to be an orthogonal set on the interval 
t <t=<tif 


| - b,()8(0) dt = . : i (7.74) 


where @*(t) denotes the complex conjugate of @,,(f). 


Verify that the set of complex exponential functions 
{err (n=O, 41, 42, 3, 2.4) 


used in the complex representation of the Fourier series is an orthogonal set on the 
interval 0 < ¢ < 27. 


First, 


ar T | 2T 
| elimT 1 dt ae a” = 0 (n x 0) 
jnt 0 


0 


since e”"" = e° = 1. Secondly, 


2T 2T 2T 
eimlT (ei)* dt= edna = wom" =0 (n x m) 


j ‘ j(n-m)t 
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7.7.2 


and, when n = m, 


2T 2T 
| eb (ginmiT) x dt = | 1 dt = oT 


0 0 


2T 
| e”™T 1 dt=0 (n#0) 


ar 
ein (g inmilT dt= 0 (n#m) 
0 2T (n=m) 


and, from (7.74), the set is an orthogonal set on the interval 0 S ¢ S 27. 


The trigonometric and exponential sets are examples of orthogonal sets that we have 
already used in developing the work on Fourier series. Examples of other sets of ortho- 
gonal functions that are widely used in practice are Legendre polynomials, Bessel func- 
tions, Hermite polynomials, Laguerre polynomials, Jacobi polynomials, Tchebyshev 
(sometimes written as Chebyshev) polynomials and Walsh functions. Over recent years 
wavelets are another set of orthogonal functions that have been widely used, particularly 
in applications such as signal processing and data compression. 


Generalized Fourier series 


Let {@,(¢)} be an orthogonal set on the interval t, < ¢ S ¢, and suppose that we wish to 
represent the piecewise-continuous function f(f) in terms of this set within this interval. 
Following the Fourier series development, suppose that it is possible to express f(f) as 
a series expansion of the form 


io x €,9,(0) (7.75) 


n=1 


We now wish to determine the coefficients c,, and to do so we again follow the Fourier 
series development. Multiplying (7.75) throughout by @,,(f) and integrating term by 
term, we obtain 


1) 


[ roasoa= S| 00.0 


n=1 ty 


which, on using (7.72) and (7.73), reduces to 


| roeoar= en 


ty 


giving 


gs = | “fldo,(t)dt (n= 1,2,3,...) (7.76) 
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Summary 


Summarizing, if f(t) is a piecewise-continuous function on the interval tf, St<4 
and {@,(f)} is an orthogonal set on this interval then the series 


KO =F c,6,(0 
n=l 
is called the generalized Fourier series of /(¢) with respect to the basis set {@,(2)}, 
and the coefficients c,, given by (7.76), are called the generalized Fourier coeffi- 
cients with respect to the same basis set. 


A parallel can be drawn between a generalized Fourier series expansion of a function 
f(t) with respect to an orthogonal basis set of functions {@,(f)} and the representation 
of a vector fin terms of an orthogonal basis set of vectors v,, v,,... , Vv, aS 


f= Gv, +...+ Gy, 


where 


_ fu fv 
a; == = 
0,0; |v; | 


There is clearly a similarity between this pair of results and the pair (7.75)-(7.76). 


Convergence of generalized Fourier series 


As in the case of a Fourier series expansion, partial sums of the form 


N 


FO = Y enG,(0 (7.77) 


n=1 


can be considered, and we wish this representation to be, in some sense, a ‘close 
approximation’ to the parent function f(t). The question arises when considering such 
a partial sum as to whether choosing the coefficients c, as the generalized Fourier 
coefficients (7.76) leads to the ‘best’ approximation. Defining the mean square error 
Ey between the actual value of f(t) and the approximation F,(¢) as 


7) 
Ey=— | [fle — Fy(OF dt 
t,- ty t, 
it can be shown that £, is minimized, for all N, when the coefficients c, are chosen 
according to (7.76). Thus in this sense the finite generalized Fourier series gives the best 
approximation. 

To verify this result, assume, for convenience, that the set {@,(4)} is orthonormal, 
and consider the Nth partial sum 


N 


FuO= ¥ 66,0 


n=1 
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where the ¢, are to be chosen in order to minimize the mean square error Ey. Now 


ty n=1 


(tf - t))Ey = | [1 = deeuo dt 


= | Fo a-23 af fnoioarr sel oo dt 


ty t n=l ty 
7) ; N N : 

= P(jd-2) &e, +> 6, 
ty n=1 n=1 


since {@,(t)} is an orthonormal set. That is, 


7) N N 
(t= t)E, = | fai y ty Gaay 
ty n n=1 


=1 


which is clearly minimized when ¢, = c,,. 


(7.78) 


Taking ¢, =, in (7.78), the mean square error Ey in approximating f(t) by Fy(t) of 


(7.73) is given by 
1 [ 2 8 
Ey =——— f(@odt- Sc, 
N th= ty , 2 
if the set {@,()} is orthonormal, and is given by 
7) N 
Ey = | fH at- > 140% 
fi 


t-t 


n=1 


if the set {,()} is orthogonal. 
Since, by definition, Ey is non-negative, it follows from (7.79) that 


7) N 
[ Poa Yr 


ty n=1 


(7.79) 


(7.80) 


a result known as Bessel’s inequality. The question that arises in practice is whether or 


not Ey > 0 as N > », indicating that the sum 


Y &n( 


n=1 


converges to the function /(¢). If this were the case then, from (7.79), 


| fwOdt= > ne 


1 n=1 


(7.81) 


which is the generalized form of Parseval’s theorem, and the set {@,(t)} is said to 
be complete. Strictly speaking, the fact that Parseval’s theorem holds ensures that the 
partial sum F\(t) converges in the mean to the parent function f(t) as N > ©, and 
this does not necessarily guarantee convergence at any particular point. In engineer- 
ing applications, however, this distinction may be overlooked, since for the functions 
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met in practice convergence in the mean also ensures pointwise convergence at points 
where f(t) is convergent, and convergence to the mean of the discontinuity at points 
where /(f) is discontinuous. 


The set {1, cost, sint,..., cosnt, sinnt} is a complete orthogonal set in the interval 
d<t<d+2n. Following the same argument as above, it is readily shown that for a 
function f(¢) that is piecewise-continuous on d < ¢ < d+ 2m the mean square error 
between /(f) and the finite Fourier series 


N N 
Fy(t) = 4a) + > G, COS nt + dS 6, sinnt 
n=1 n=1 


is minimized when 4), a, and 5, (n= 1, 2,3, . . . ) are equal to the corresponding Fourier 
coefficients dp, a, and b, (n= 1, 2,3, ...) determined using (7.4) and (7.5). In this case 
the mean square error Ey is given by 


d+20 N 
eh! | f(t) dt m| $a9 + ¥ (ay + by) 


2m d n=1 


Bessel’s inequality (7.80) becomes 


d+2n N 
| f(t) dt =m] $40 + Y (an + by) 


d n=1 


and Parseval’s theorem (7.81) reduces to 
1 d+20 és 
a 2 Let 2 1 2 2 
on [ f (t) aot 2 D (an bs) 


which conforms with (7.62). Since, in this case, the basis set is complete, Parseval’s 
theorem holds, and the Fourier series converges to f(t) in the sense discussed above. 


7.7.4 Exercises 


The Fourier series expansion for the periodic square Determine the mean square error corresponding to 
wave approximations to f(t) based on the use of one term, 
two terms and three terms respectively in the series 
-1 (-<t<0 expansion. 
joa. 
1 (0<t<m7) ; 
41 = The Legendre polynomials P,(f) are generated by 
f(t+ 2m) =f the formula 


is 


_¥ 4 _ 
ft= 2 Toro sin(2n - 1)t 


pQ=tLe-1 @=0,1,2,...) 
2"n! 


t" 
and satisfy the recurrence relationship 


P(t) = (2n — Ith) — (a — DFO) 
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42 


43 


(a) Deduce that 
P(j=1, P\j=t 
Pt) =$(30?-1), P,(d) = } (SP - 3) 
(b) Show that the polynomials form an orthogonal 
set on the interval (—1, 1) and, in particular, that 


| POE, dt 


= fe (n #m) 
2/)(2n+1) (n=m;m=0,1,2,...) 
(c) Given that the function 


-1 (-l<t<0) 
S®=4 0 (t=0) 
1 (0<t<1) 


is expressed as a Fourier—Legendre series 
expansion 


f= ¥ 6PM 


r=0 
determine the values of co, c,, C, and c;3. 
(d) Plot graphs to illustrate convergence of the 
series obtained in (c), and compare the mean 


square error with that of the corresponding 
Fourier series expansion. 


Repeat parts (c) and (d) of Exercise 41 for the 
function 


fee) = {° 


(123= 0) 
(0<x<1) 


Laguerre polynomials L,(t) are generated by the 
formula 


L@=¢LVe) @=0,1,2,...) 
df” 


and satisfy the recurrence relation 


L(t) =(2n-1-dOL,..(0 -(n 


1V°L,_.() 
(n=2,3,...) 


These polynomials are orthogonal on the 
interval 0 < t < with respect to the weighting 
function e“, so that 


| e'L,(t)L,,(t) dt = .’ 
" (nl) 


(n #m) 


(n=m) 


44 


(a) Deduce that 
L(t)=1, L()=1-t 
L,(t)=2-4t+0 
L,(t) = 6 — 18t+ 97? - #7 
(b) Confirm the above orthogonality result in the 
case of Lo, L,, L, and L;. 


(c) Given that the function /(¢) is to be 
approximated over the interval 0 < t < © by 


fO=¥ ¢ L(t) 


r=0 


show that 


c= : | f(t) e"L,(t) dt 
(rl) Jo 


(r=0,1,2,...) 


(Note: Laguerre polynomials are of particular 
importance to engineers, since they can 
be generated as the impulse responses of 
relatively simple networks.) 


Hermite polynomials H,(t) are generated by the 
formula 


aCe obs ef? dad et? 


(n=0,1,2,...) 
and satisfy the recurrence relationship 
A(t) = tH, (t) — (n — I)H,,o(0) 

(n= 2, 3,...) 


The polynomials are orthogonal on the interval 
—co < ¢ < co with respect to the weighting 
function e””, so that 


| eHpndn ar=| ° 
a \(2n)n! 


(n#m) 
(n=m) 
(a) Deduce that 

H(t)=1, H()=t 
A(j=?-1, A)=t-3t 
H(t) = t* — 61? +3 


(b) Confirm the above orthogonality result for 
Ay, H,, H, and H3. 
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(c) Given that the function /(7) is to be 
approximated over the interval —co < t < oo by 


f= > eA) 
show that 
cote | oP) H,(t) dt 
rhyme) 


$= 0 Asin) 


Tchebyshev polynomials 
formula 


T,(t) are generated by the 


T,(t)=cos(ncos't) (n=0,1,2,...) 


or 
[n/2] nl 
TAO = 3M Giga dF 
(n=0, 1,2,...) 
where 
[n/2] = n/2 (even 7) 
(n-1)/2 (odd n) 


They also satisfy the recurrence relationship 


T,() (n = 2, 3, areiie ) 


and are orthogonal on the interval-1 <t< 1 
with respect to the weighting function 1/\(1 — f°), 
so that 


1 0 
| TOTO Ns, 
4-1 =#) ° 


tT (m=n=0) 


T(t) = 21T,(é) - 


(m #n) 
(m=n#0) 


(a) Deduce that 


T(AO=1, TH=t 
T,(t)=2t?-1, 1,(f) = 40° — 3¢ 
T(t) = 8t* — 80° +1 


T(t) = 16¢° — 208° + 5t 


(b) Confirm the above orthogonality result for 
Ty, T,, T, and T;. 
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(c) Given that the function (4) is to be 
approximated over the interval —1 < ¢ < | by 


f= > oT 


r=0 


show that 
coo! | LOT 
mr} (=P) 


<=?) MOTO a, (21,2...) 
T alle?) 


With developments in digital techniques, Walsh 
functions W,(t) have become of considerable 
importance in practice, since they are so easily 
generated by digital logic circuitry. The first four 
Walsh functions may be defined on the interval 
0<t<Tby 


W(t) = (0<t<T) 


\T 
/ WyT (0<t<3T) 


W(t) = 
WjT GT <t<T) 


WAt) = ea (0<¢<17,37<1t<T) 
Wu? Gr<1< Gr) 
W(t) = 
oe (0<¢<i7,27<1t<i7,17<t<T) 
=f Gra1<e7, r<r< ir) 


(a) Plot graphs of the functions W,(t), W,(t), W(t) 
and W,(t), and show that they are orthonormal 
on the interval 0 = ¢ < T. Write down an 
expression for W,(t). 

(b) The Walsh functions may be used to obtain 
a Fourier—Walsh series expansion for a 
function /(‘), over the interval0O <¢S 7, 
in the form 


f= > WA) 
r=0 
Illustrate this for the square wave of 
Exercise 40. What is the corresponding mean 
square error? Comment on your answer. 
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7.8 Engineering application: GES yitemilitadeats 


Many control systems containing a nonlinear element may be represented by the block 
diagram of Figure 7.42. In practice, describing function techniques are used to analyse 
and design such control systems. Essentially the method involves replacing the non- 
linearity by an equivalent gain N and then using the techniques developed for linear 
systems, such as the frequency response methods of Section 5.8. If the nonlinear ele- 
ment is subjected to a sinusoidal input e(t) =X sin wt then its output z(t) may be repre- 
sented by the Fourier series expansion 


Figure 7.42 Nonlinear Nonlinear element 
control system. Linear 


element 


2(t) = fay + », a, COS N@t + y, b, sin not 


n=1 n=1 


= tayt ¥ A, sin(not+ @,,) 
n=1 
with A,, = \(a, + b;) and @, = tan '(a,/b,). 
The describing function N(X ) of the nonlinear element is then defined to be the 
complex ratio of the fundamental component of the output to the input; that is, 


N(X) =a" 


with N(X) being independent of the input frequency @ if the nonlinear element is 
memory-free. 

Having determined the describing function, the behaviour of the closed-loop system 
is then determined by the characteristic equation 


1+ MX)G(jo) =0 


If a combination of XY and @ can be found to satisfy this equation then the system is 
capable of sustained oscillations at that frequency and magnitude; that is, the system 
exhibits limit-cycle behaviour. In general, more than one combination can be found, 
and the resulting oscillations can be a stable or unstable limit cycle. 

Normally the characteristic equation is investigated graphically by plotting G( ja) 
and —1/N(X), for all values of X, on the same polar diagram. Limit cycles then occur at 
frequencies and amplitudes corresponding to points of intersection of the curves. Some- 
times plotting can be avoided by calculating the maximum value of V(X) and hence the 
value of the gain associated with G(s) that will just cause limit cycling to occur. 

Using this background information, the following investigation is left as an exercise 
for the reader to develop. 
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Figure 7.43 (a) Relay; 
(b) relay with dead 
zone. 


Output 


L 


Figure 7.44 Nonlinear 
system of exercise. 


(a) 
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Show that the describing functions V(X’) and N,(X ) corresponding respectively 


to the relay (on-off nonlinearity) of Figure 7.43(a) and the relay with dead zone 


of Figure 7.43(b) are 


4L 


4L 
MS A) 


(b) 


- 


For the system of Figure 7.44 show that a limit cycle exists when the nonlinearity 


is the relay of Figure 7.43(a) with L = 1. Determine the amplitude and frequency 


of this limit cycle. 


In an attempt to eliminate the limit-cycle oscillation, the relay is replaced by 
the relay with dead zone illustrated in Figure 7.43(b), again with L = 1. Show that 
this allows our objective to be achieved provided that h > 10/31. 


7.9 Review exercises (1-20) 


1A periodic function f(A) is defined by 2 
2 
if Oka 
ft)= : e 
O << tS 20) 


S(t + 2m) =f) 


Obtain a Fourier series expansion of f(t) and 
deduce that 


Ww 
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Determine the full-range Fourier series expansion 
of the even function f(t) of period 27 defined by 


2 il 
ti (0 StS 5m) 


Ko=| 


(u-t) (nm <t<n) 


To what value does the series converge at t= in? 
A function f(¢) is defined for0 S¢S 5 T by 

T) 

if) 


<= 


t (0<t<} 
f= 
=T-t 


Ly ae 
5 Grist 


an 
2 
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Sketch odd and even functions that have a period 
T and are equal to f(t) for0 St < $7. 


(a) Find the half-range Fourier sine series of f(t). 

(b) To whe value will the series converge for 
t=-1T? 

(c) What i is the sum of the following series? 


® ara ey 


Prove that if g(x) is an odd function and f(x) an even 
function of x, the product g(x)[c + f(x)] is an odd 
function if c is a constant. 

A periodic function with period 27 is defined by 


F@) = +460 - &) 


in the interval —t = 0 < zm. Show that the Fourier 
series representation of the function is 


n+l 
F(0)= > et sin nO 
n=1 
A repeating waveform of period 27 is described by 
T+t (~<ts 
Hija i 2 = 


TU 
jh Ao = f= 1) 


1 
=i") 


Sketch the waveform over the range t = —27 to 
t = 27 and find the Fourier series representation 
of f(t), making use of any properties of the 
waveform that you can identify before any 
integration is performed. 


A function f(x) is defined in the interval 
-1 <x <1 by 


foxy = {18 (-e <x <e) 
0 (GI ea =) 


Sketch a graph of f(x) and show that a Fourier 
series expansion of f(x) valid in the interval 
—1 <x < 1 is given by 


TS 


sin NME 


COS NIX 


Show that the half-range Fourier sine series for the 
function 


no=(1-4) (0<t<n) 


10 


Tal 


is 


n 


|sin nt 


Find a half-range Fourier sine and Fourier cosine 
series for f(x) valid in the intervalO <x< 7 
when f(x) is defined by 


3 
fo) =| 
Ui S2.8 


Sketch the graph of the Fourier series obtained 
for —2m <x < 2n. 


f= 2 {1- 


n=1 nt 


Co] a) 


A function f(x) is periodic of period 27 and is 
defined by f(x) = e* (-m < x < 7). Sketch the 
graph of f(x) from x = —27 to x = 2m and prove 
that 


I(x) = 2snhe £4 Ss Coos na n sin nx) 


ferris 


A function (4) is defined on 0 < t < 1 by 
fQ=n-t 

Find 

(a) a half-range Fourier sine series, and 


(b) a half-range Fourier cosine series for f(t) 
valid forO <<t< 1. 


Sketch the graphs of the functions represented 
by each series for —-2m < t < 2m. 


Show that the Fourier series 


ty cos(2n - 1)t 
T (2n-1) 


n=l 


represents the function f(t), of period 21, 
given by 


Ol = 770) 
go=| | (n <t<0) 


Deduce that, apart from a transient component 
(that is, a complementary function that dies away 
as t > co), the differential equation 


Hex = fy) 
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is) 


has the solution 


xein-2y cos(2n - 1)t+ (2n - 1)sinQn - 1)t 
Tt 


Cro) Met Oneal 


Show that if f(4) is a periodic function of period 
2n and 


t/T 
we ee - p/n 


(0<t<T) 
(ut <t < 2m) 


then 


fry=i- ty sstns 
ne ay ((2up4P 1) 


Show also that, when @ is not an integer, 
1 
y= —,5 (1- cos wt) 
20 


= ee 
(Qn+1) lo -(2n+1) ] 


n=1 


satisfies the differential equation 


Gye dy =f) 


subject to the initial conditions y = dy/dt = 0 at 
t=0. 


(a) A periodic function f(t), of period 27, is 
defined in -m < t = 1 by 


ft) = e 
if 


Obtain a Fourier series expansion for f(t), and 
from it, using Parseval’s theorem, deduce that 


ed reer: 


(b) By formally differentiating the series obtained 


in (a), obtain the Fourier series expansion of 
the periodic square wave 
=1 (r=t<= 0) 
s)=) 0 (t=0) 
I (OS <a) 


g(t + 2m) = g(t) 
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Check the validity of your result by 
determining directly the Fourier series 
expansion of g(f). 


A periodic function /(‘), of period 27, is defined 
in the range -m < t < 7 by 


f(t) = sin St 


Show that the complex form of the Fourier series 
expansion for f(¢) is 


Aa) = > rer 


(a) Find the Fourier series expansion of the 
voltage v(t) represented by the half-wave 
rectified sine wave 


(Ome) 


ie sin(2nt/T) 
v(t) = 
Gr<re<T 


v(t + T) = (2) 


(b) If the voltage v(f) in (a) is applied to a 
10 Q resistor, what is the total average power 
delivered to the resistor? What percentage 
of the total power is carried by the second- 
harmonic component of the voltage? 


The periodic waveform f(t) shown in Figure 7.45 
may be written as 


mn —-4n —3n -2n -2 


Figure 7.45 Waveform f(t) of Review 
exercise 16. 


fO=14+ gH 
where g(t) represents an odd function. 
(a) Sketch the graph of g(?). 
(b) Obtain the Fourier series expansion for g(¢), 


and hence write down the Fourier series 
expansion for f(¢). 


Show that the complex Fourier series expansion 
for the periodic function 


f=t O<t<2n) 
f(t + 2m) =f 


636 


18 


19 


FOURIER SERIES 


Ayers y 1 


n=-090 


n#0 


(a) A square-wave voltage v(t) of period T is 
defined by 


v(t) an 
1 


vu(t+ T)=v() 


(-}T <t<0) 
(0<t<$T) 


Show that its Fourier series expansion is 
given by 


_4~— sin[(4n-2)nt/T] 
JO SOD aaa 


n=1 


(b) Find the steady-state response of the circuit 
shown in Figure 7.46 to the sinusoidal input 
voltage 


v(t) = sin ot 


and hence write down the Fourier series 
expansion of the circuit’s steady-state response 
to the square-wave voltage v(f) in (a). 


19 


Figure 7.46 Circuit of Review exercise 18. 


u(t} (~) 


1H 


(a) Defining the nth Tchebyshev polynomial by 
T,(t) = cos(n cos" f) 


use Euler’s formula cos 6 = } (e'” + e7!*) 
to obtain the expansions of ¢* and 17" 
in Tchebyshev polynomials, where k is a 
positive integer. 


(b) Establish the recurrence relation 
T(t) = 2tT,_() — Ta) 
(c) Write down the values of 7,(¢) and 7,(t) from 


the definition, and then use (b) to find 7;(t) and 


T;(t). ; 
(d) Express ¢° — 5t* + 7° + 6¢ — 8 in Tchebyshev 
polynomials. 


20 


(e) Find the cubic polynomial that approximates 
to 


P—5t*+ 7+ 6t—8 


over the interval (—1, 1) with the smallest 
maximum error. Give an upper bound for 
this error. Is there a value of t for which this 
upper bound is attained? 


The relationship between the input and output of 
a relay with a dead zone A and no hysteresis is 


shown in Figure 7.47. Show that the describing 
function is 


wor StI] 


for an input amplitude x;. 


Output M 


Figure 7.47 Relay with dead zone of Review 
exercise 20. 


If this relay is used in the forward path of 
the on—off positional control system shown in 
Figure 7.48, where the transfer function 


re ee 
s(T,s + 1) (Ths +1) 


characterizes the time constant of the servo-motor, 
and the inertia and viscous damping of the load, 
show that a limit-cycle oscillation will not occur 
provided that the dead zone in the relay is such 
that 


4MK 1,T, 
un 1,4+T; 


AN 


Figure 7.48 Positional control system of Review 
exercise 20. 
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8.1 


8.2 


8.2.1 


Introduction 


In Chapter 7 we saw how Fourier series provided an ideal framework for analysing the 
steady-state response of systems to a periodic input signal. In this chapter we extend 
the ideas of Fourier analysis to deal with non-periodic functions. We do this through 
the introduction of the Fourier transform. As the theory develops, we shall see how the 
complex exponential form of the Fourier series representation of a periodic function 
emerges as a special case of the Fourier transform. Similarities between the transform 
and the Laplace transform, discussed in Chapter 5, will also be highlighted. 

While Fourier transforms first found most application in the solution of partial 
differential equations, it is probably true to say that today Fourier transform methods 
are most heavily used in the analysis of signals and systems. This chapter is therefore 
developed with such applications in mind, and its main aim is to develop an understand- 
ing of the underlying mathematics as a preparation for a specialist study of application 
areas in various branches of engineering. 

Throughout this book we draw attention to the impact of digital computers on engin- 
eering and thus on the mathematics required to understand engineering concepts. While 
much of the early work on signal analysis was implemented using analogue devices, the 
bulk of modern equipment exploits digital technology. In Chapter 5 we developed the 
Laplace transform as an aid to the analysis and design of continuous-time systems 
while in Chapter 6 we introduced the z and & transforms to assist with the analysis and 
design of discrete-time systems. In this chapter the frequency-domain analysis intro- 
duced in Chapter 5 for continuous-time systems is consolidated and then extended to 
provide a framework for the frequency-domain description of discrete-time systems 
through the introduction of discrete Fourier transforms. These discrete transforms pro- 
vide one of the most advanced methods for discrete signal analysis, and are widely used 
in such fields as communications theory and speech and image processing. In practice, 
the computational aspects of the work assume great importance, and the use of appro- 
priate computational algorithms for the calculation of the discrete Fourier transform is 
essential. For this reason we have included an introduction to the fast Fourier transform 
algorithm, based on the pioneering work of J. W. Cooley and J. W. Tukey published 
in 1965, which it is hoped will serve the reader with the necessary understanding for 
progression to the understanding of specialist engineering applications. 

An additional engineering application section has been included in this new edition. 
In this we discuss the discrete-time Fourier transform to provide the means of describ- 
ing the so-called direct design method for digital filters which is based on the use of the 
desired frequency response, without using an analogue prototype design. This naturally 
leads to considering ‘windowing’ and a brief introduction to this topic is included. 


The Fourier transform 


The Fourier integral 


In Chapter 7 we saw how Fourier series methods provided a technique for the 
frequency-domain representation of periodic functions. As indicated in Section 7.6.3, 
in expressing a function as its Fourier series expansion we are decomposing the function 
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Window 


Figure 8.1 The view of f(7) through a window of Figure 8.2 The periodic function g(t) based on the 
length T. ‘windowed’ view of f(t). 


into its harmonic or frequency components. Thus a periodic function f(4), of period 7’, 
has frequency components at discrete frequencies 


ie ann ano =0:1,9,3.....5 

where @, is the fundamental frequency, that is the frequency of the parent function /(?). 
Consequently we were able to interpret a Fourier series as constituting a discrete fre- 
quency spectrum of the periodic function f(d), thus providing an alternative frequency- 
domain representation of the function to its time-domain waveform. However, not all 
functions are periodic and so we need to develop an approach that will give a similar 
representation for non-periodic functions, defined on —co < ¢ < oo, One way of achiev- 
ing this is to look at a portion of a non-periodic function f(4) over an interval 7, by 
imagining that we are looking at a graph of f(A through a ‘window’ of length 7, and 
then to consider what happens as T gets larger. 

Figure 8.1 depicts this situation, with the window placed symmetrically about the 
origin. We could now concentrate only on the ‘view through the window’ and carry out 
a Fourier series development based on that portion of f(t) alone. Whatever the beha- 
viour of f(f) outside the window, the Fourier series thus generated would represent the 
periodic function defined by 


2 ie (It] < 37) 
f(it—nT) (Qn- DT <|t| < $Qn+ 17) 


Figure 8.2 illustrates g(t), and we can see that the graphs of f(t) and g(t) agree on the 
interval (—} 7, } 7). Note that this approach corresponds to the one adopted in Section 
7.3 to obtain the Fourier series expansion of functions defined over a finite interval. 

Using the complex or exponential form of the Fourier series expansion, we have 
from (7.53) and (7.57) that 


g(t)= 9 G,e"" (8.1) 


with 


T/2 
Ge | g(t)e "dt (8.2) 


-T/2 
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and where 
Oy = 20/T (8.3) 


Equation (8.2) in effect transforms the time-domain function g(f) into the associated 
frequency-domain components G’,, where 7 is any integer (positive, negative or zero). 
Equation (8.1) can also be viewed as transforming the discrete components G,, in the 
frequency-domain representation to the time-domain form g(f). Substituting for G, in 
(8.1), using (8.2), we obtain 


a= > E | ain w - (8.4) 


nace -T/2 
The frequency of the general term in the expansion (8.4) is 


2nn _ 


T NO) = O, 


and so the difference in frequency between successive terms is 
21 21 
= ae 1 = -_-_ == AO 
T [(n aH T 


Since A@= @,, we can express (8.4) as 


os: T/2 
= 1 -j@,T jO@nt 
th= — e "Aa 8.5 
g(t) Sz] g(te ; (8.5) 
" -T/2 
Defining G(jq@) as 
7/2 
G(j@) = | 2(T) et dt (8.6) 
-T/2 
we have 
_ 25 = —j@yt 
g(t) = on by e "G(j@,)A@ (8.7) 


n=—0o 


As T > ©, our window widens, so that g(t) = f(t) everywhere and A@ — 0. Since we 
also have 


co 


: ag 1 jt a, - _ 25 jot A, 
jim. = » e"G(j@,)A@ = +| e'"G(ja) da 


n=—co 


it follows from (8.7) and (8.6) that 


fh = | 2 é| f(t) mas do (8.8) 
_|2% me 


The result (8.8) is known as the Fourier integral representation of f(t). A set of 
conditions that are sufficient for the existence of the Fourier integral is a revised form 
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of Dirichlet’s conditions for Fourier series, contained in Theorem 7.2. These conditions 
may be stated in the form of Theorem 8.1. 


Dirichlet’s conditions for the Fourier integral 
If the function f(A) is such that 
(a) it is absolutely integrable, so that 


| | f(t) | dt < ec 


(that is, the integral is finite), and 
(b) it has at most a finite number of maxima and minima and a finite number of 
discontinuities in any finite interval 


then the Fourier integral representation of f(t), given in (8.8), converges to /(f) at all 
points where f(t) is continuous and to the average of the right- and left-hand limits of 


S(t) where f(#) is discontinuous (that is, to the mean of the discontinuity). 


end of theorem 


As was indicated in Section 7.2.9 for Fourier series, the use of the equality sign in 
(8.8) must be interpreted carefully because of the non-convergence to f(t) at points of 
discontinuity. Again the symbol ~ (read as ‘behaves as’ or ‘represented by’) rather than 
= is frequently used. 

The absolute integrable condition (a) of Theorem 8.1 implies that the absolute area 
under the graph of y = f(A) is finite. Clearly this is so if f(t) decays sufficiently fast with 
time. However, in general the condition seems to imply a very tight constraint on 
the nature of f(¢), since clearly functions of the form f(‘) = constant, f(t) =e“, f() =e, 


f(t) = sin wt, and so on, defined for -c0o < t < oo, do not meet the requirement. In 


practice, however, signals are usually causal and do not last for ever (that is, they only 
exist for a finite time). Also, in practice no signal amplitude goes to infinity, so con- 
sequently no practical signal /(t) can have an infinite area under its graph y = f(¢). Thus 
for practical signals the integral in (8.8) exists. 

To obtain the trigonometric (or real) form of the Fourier integral, we substitute 


ear) = cos O(T— t) -j sin O(T— t) 


in (8.8) to give 
fhe +| | f(D)lcos a(t — t) —jsinw(t — 1) J dt do 


Since sin @(T — f) is an odd function of @, this reduces to 


co 


fin= bl | F(T) cos w(t — t) dt da 
2m} J 


which, on noting that the integrand is an even function of @, reduces further to 


fit) =* | da | f(t) cos w(t - tdt (8.9) 
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The representation (8.9) is then the required trigonometric form of the Fourier 
integral. 

If f(#) is either an odd function or an even function then further simplifications of (8.9) 
are possible. Detailed calculations are left as an exercise for the reader, and we shall 
simply quote the results. 


(a) If f(A) is an even function then (8.9) reduces to 


HOS - | | f(t) cos ot cos wt dt do (8.10) 


which is referred to as the Fourier cosine integral. 


(b) Iff(A is an odd function then (8.9) reduces to 


HOVE : | | f(t) sin ot sin ot dt do (8.11) 


which is referred to as the Fourier sine integral. 


In the case of the Fourier series representation of a periodic function it was a matter 


of some interest to determine how well the first few terms of the expansion represented 
i the function. The corresponding problem in the non-periodic case is to investigate how 
well the Fourier integral represents a function when only the components in the lower 
part of the (continuous) frequency range are taken into account. To illustrate, consider 
-] e 1! the rectangular pulse of Figure 8.3 given by 
Figure 8.3 Rectangular 
<= 
pulse f=! (It] <1) 
no = |) (\t| <1) 0 ({t| > 1) 
, 0 (\t)>1)- 


This is clearly an even function, so from (8.10) its Fourier integral is 


ar s 
S(t) = e l cos wt cos wt dt d@ = 2] cos ¢ OtsSIn@ a 
m 0 TU ‘i @ 


0 
An elementary evaluation of this integral is not possible, so we consider frequencies 
@ < @», when 


@ 


f(t) = 2) Oe Otsin® , 


@ 
0 


= Lj mt DD aa = 1| ’ sin o(t — a 
Tt @ Tt @ 


0 0 


Ott) ; Ol-l) , 
sin u sin u 
—du-- — du 
u Tt u 


0 0 


www.2Ofile.org 


8.2 THE FOURIER TRANSFORM 643 


The integral 


sic) =| 0) 
u 


0 


occurs frequently, and it can be shown that 


n_2n+1 


Eee = = Il) 
ee 2 (Qn + 1)(2n+ 1)! 


Its values have been tabulated (see for example L. Rade and B. Westergren, Beta 
Mathematics Handbook, Chartwell-Bratt Ltd, Bromley, Kent, 1990). Thus 


S(O = Si(@o(t + 1) — Si(@o(t — 1) (8.12) 


This has been plotted for @) = 4, 8 and 16, and the responses are shown in Figures 8.4(a), 
(b) and (c) respectively. Physically, these responses describe the output of an ideal 
low-pass filter, cutting out all frequencies @ > @», when the input signal is the rectan- 
gular pulse of Figure 8.3. The reader will no doubt note the similarities with the 
Fourier series discussion of Section 7.2.9 and the continuing existence of the Gibbs 
phenomenon. 


Figure 8.4 

Plot of (8.12): 

(a) @) =4; (b) @ = 8; 
(c) @ = 16. 
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8.2.2 The Fourier transform pair 


We note from (8.6) and (8.7) that the Fourier integral (8.8) may be written in the form 
of the pair of equations 


F(jo) = | Wie. de (8.13) 
ro= | F(jo)e’ dw (8.14) 


F(j@) as defined by (8.13) is called the Fourier transform of f(t), and it provides 
a frequency-domain representation of the non-periodic function f(f), whenever the 
integral in (8.13) exists. Note that we have used the notation F(j@) for the Fourier trans- 
form of f(t) rather than the alternative F(@), which is also in common use. The reason 
for this choice is a consequence of the relationship between the Fourier and Laplace 
transforms, which will emerge later in Section 8.4.1. We stress that this is a choice that 
we have made, but the reader should have no difficulty in using either form, provided 
that once the choice has been made it is then adhered to. Equation (8.14) then provides 
us with a way of reconstructing f(¢) if we know its Fourier transform F(jq). 

A word of caution is in order here regarding the scaling factor 1/27 in (8.14). 
Although the convention that we have adopted here is fairly standard, some authors 
associate the factor 1/27 with (8.13) rather than (8.14), while others associate a factor 
of (2x)? with each of (8.13) and (8.14). In all cases the pair combine to give the 
Fourier integral (8.8). We could overcome this possible confusion by measuring the 
frequency in cycles per second or hertz rather than in radians per second, this being 
achieved using the substitution f= @/2n, where fis in hertz and @ is in radians per 
second. We have not adopted this approach, since @ is so widely used by engineers. 

In line with our notation for Laplace transforms in Chapter 5, we introduce the 
symbol # to denote the Fourier transform operator. Then from (8.13) the Fourier transform 
F{ f(t)} of a function f(A) is defined by 


Fi f(t)} = F(jo) = | f(t)e?” dt (8.15) 


whenever the integral exists. Similarly, using (8.14), we define the inverse Fourier 
transform of G(j@) as 


F ' {G (jo)} = g(t) = | G(ja) e'' do (8.16) 


whenever the integral exists. The relations (8.15) and (8.16) together constitute the 
Fourier transform pair, and they provide a pathway between the time- and frequency- 
domain representations of a function. Equation (8.15) expresses f(t) in the frequency 
domain, and is analogous to resolving it into harmonic components with a continuously 
varying frequency w. This contrasts with a Fourier series representation of a periodic 
function, where the resolved frequencies take discrete values. 
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Solution 


Example 8.2 


Solution 
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The conditions for the existence of the Fourier transform F(jq@) of the function f(t) 
are Dirichlet’s conditions (Theorem 8.1). Corresponding trigonometric forms of the 
Fourier transform pair may be readily written down from (8.9), (8.10) and (8.11). 


Does the function 
f)=1 (Ce <t<e) 


have a Fourier transform representation? 


Since the area under the curve of y = f(t) (-co < ft < ce) is infinite, it follows that 
J=..| f®|d¢ is unbounded, so the conditions of Theorem 8.1 are not satisfied. We can 
confirm that the Fourier transform does not exist from the definition (8.15). We have 


2 a 
| le?” dt = lim | edt 
oes ao a 


= lim ae _ a) 


O00 jo 
= lim 2Sin@a 
aoe @ 


Since this last limit does not exist, we conclude that f(f) = 1 (-co < t < ©) does not 
have a Fourier transform representation. 


It is clear, using integration by parts, that f(t) = t (-co < ¢ < ee) does not have a 
Fourier transform, nor indeed does f(t) = t" (n > 1, an integer; —co < t < co), While 
neither e“ nor e“ (a > 0) has a Fourier transform, when we consider the causal signal 
f(t) = H(t)e™ (a > 0), we do obtain a transform. 


Find the Fourier transform of the one-sided exponential function 
fQ=HOe™ (a>) 


where /(f) is the Heaviside unit step function. 


The graph of f(f) is shown in Figure 8.5, and we can show that the area under the graph 
is bounded. Hence, by Theorem 8.1, a Fourier transform exists. Using the definition 
(8.15), we have 


Fi f(t} =| H(the“e'dt (a> 0) 


z= | eatin dy - ie baa 
: a+ jo), 
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Figure 8.5 fy 
The ‘one-sided’ 
exponential function 
fl) = HH“ 

(a> 0). 


O t 


so that 


1 
a+j@ 


FLH(the “y= (8.17) 


Example 8.3 — Calculate the Fourier transform of the rectangular pulse 


_fA (tl<T) 
fo={' (\t| > T) 


Solution The graph of f(t) is shown in Figure 8.6, and since the area under it is finite, a Fourier 
transform exists. From the definition (8.15), we have 


f()) : 
: Ff} -| Ae? dt = ere a 
- 2A w= 0 
—— Oo 4 = 2AT sinc oT 


Figure 8.6 The 


where sinc x is defined, as in Example 7.22, by 
rectangular pulse 


< sin 
n= |" ee : = (x #0) 
0 (ltl >T7)- sinc x = X 
1 (x = 0) 
Figure 8.7 
A brief table of «= 
Fourier transforms. f(t) FL f(t)} = | fe at 
—aty poe ees 
e*H(t) (a> 0) ae 
te*H(t) (a> 0) ee 
(a + jo) 
ie CS 2) 2AT sincoT 
OQ Cele ©) 
e* (a> 0) ue 
ato 
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By direct use of the definition (8.15), we can, as in Examples 8.2 and 8.3, determine 
the Fourier transforms of some standard functions. A brief table of transforms is given 
in Figure 8.7. 


In MATLAB, incorporating the Symbolic Math Toolbox, the Fourier transform F(ja@) 
of f(t) is obtained using the commands 


syms wt 
ig=se(oybealeue (Ge (je) pie 5 iy) 


whilst the inverse Fourier transform f(t) of F(j@) is obtained using the command 
f=ifourier(F(jw),w,t) 
Corresponding commands in MAPLE are 


with(inttrans) : 

Peitouiedee (1 (iE) , E,W) F 

f=invfourier(F(jw),w,t); 

Returning to Example 8.2, and considering the particular case of a = 2, the 

commands 

syms w t 

H=sym(‘’Heaviside(t)’); 

F=fourier (H*exp (-2*t) ) 
in MATLAB return 

etal // (2seaL Ay) 
as expected. In MATLAB there is an assume command (as in MAPLE) to enable 
us to specify that a > 0. However, since abs(a) = a for a > 0, the following commands 
in MATLAB can be used to deal with the general case 


syms wta 
H=sym(‘Heaviside(t)’); 
F=fourier (H*exp(-abs (a) *t),t,w) 


As another illustration, consider the function f(t) =e", a > 0, given in the 
table of Figure 8.7. Considering the particular case a = 2 then the MATLAB 
commands 


syms w t 
F=fourier (exp (-2*abs(t),t,w) 


return 
F=4/ (4+w%2) 


as specified in the table. It is left as an exercise to consider the general case of a. To 
illustrate the use, in MATLAB, of the ifourier command this transform can be 
inverted using the commands 


syme: Ww 
f=ifourier (4/ (4+w%2) ,w,t) 


which return 
f=Heaviside(t) *exp(-2*t)+exp(2*t) *Heaviside(-t) 


which corresponds to the expected answer f= exp(—2*abs(f)). 
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8.2.3 


Example 8.4 


Solution 


As another illustration consider the Fourier transform F(@) = 1/(a + j@)° given in 
the second entry of the table in Figure 8.7. The MATLAB commands 


syms wta 
f=ifourier (1/ (a+i*w) *2,w,t) 


return 
f=t*exp(-a*t) *Heaviside(t) 
as given in the table. 


Considering the rectangular pulse f(t) of Example 8.3, we first express the pulse 
in terms of Heaviside functions as 


ff) =A(A(t+ T) - H(t - T)) 
and then use the MATLAB commands 


syms wt TA 
H=sym (‘Heaviside (t+T) -Heaviside(t-T)’); 


i Sse@neseke (UNIS, 1c , Wi) 5 
F=simple(F) 


which return 


F=2*A*sin(T*w) /w 


The continuous Fourier spectra 


From Figure 8.7, it is clear that Fourier transforms are generally complex-valued func- 
tions of the real frequency variable w. If #{f(t)} = F(j@) is the Fourier transform of 
the signal f(t) then F'(jq@) is also known as the (complex) frequency spectrum of /(f). 
Writing F'(j@) in the exponential form 


F(j@) = |F (jo) e'*8"0 
plots of |F(j@)| and arg F(j@), which are both real-valued functions of @, are called the 


amplitude and phase spectra respectively of the signal f(t). These two spectra repres- 
ent the frequency-domain portrait of the signal f(t). In contrast to the situation when 


f(t) was periodic, where (as shown in Section 7.6.3) the amplitude and phase spectra 


were defined only at discrete values of @, we now see that both spectra are defined for 
all values of the continuous variable a. 


Determine the amplitude and phase spectra of the causal signal 
fQ=e"H(t) (a> 0) 
and plot their graphs. 


From (8.17), 
F{ fit)} = F(jo) = aa 


Thus the amplitude and argument of F'(j@) are 
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| F(jo)| = —-. 
\(a + @) 


(8.18) 


arg F(j@) = tan'(1) — tan'(2) = ~tan'(2) (8.19) 


These are the amplitude and phase spectra of f(t), and are plotted in Figure 8.8. 


Figure 8.8 | Fijw)| 
(a) Amplitude and 

(b) phase spectra of the 
one-sided exponential 
function f(t) = e“H(t) 
(a > 0). 


V/a 


oO w 


Generally, as we have observed, the Fourier transform and thus the frequency spec- 
trum are complex-valued quantities. In some cases, as for instance in Example 8.3, the 
spectrum is purely real. In Example 8.3 we found that the transform of the pulse illus- 
trated in Figure 8.6 was 


F(j@) = 2AT sinc oT 
where 
sin oT 


sinc OT = oT 
1 (@ = 0) 


(@ # 0) 


is an even function of @, taking both positive and negative values. In this case the 
amplitude and phase spectra are given by 


|F(ja)| = 2AT |sinc @T| (8.20) 
i = 
magac ere’ (8.21) 
T (sinc @7T < 0) 


with corresponding graphs shown in Figure 8.9. 


www.2Ofile.org 


650 THE FOURIER TRANSFORM 


Figure 8.9 | Fjw)l 
(a) Amplitude and 
(b) spectra of the pulse 2AIT. 
A t]) ST 
AO = (ltl <7) 
0 (\|t) >T).- 
—3n/T —2n/T —n/T O nT 2n/T 3n/T 
(a) 
arg F(jw) 
‘18 
8) w 
(b) 
Figure 8.10 


Frequency spectrum 
(real-valued) of the pulse 
A (|t|) ST) 


Ko ={) (\t| > T7)- 


In fact, when the Fourier transform is a purely real-valued function, we can plot all 
the information on a single frequency spectrum of F(j@) versus @. For the rectangular 
pulse of Figure 8.6 the resulting graph is shown in Figure 8.10. 

From Figure 8.7, we can see that the Fourier transforms discussed so far have 
two properties in common. First, the amplitude spectra are even functions of the 
frequency variable @. This is always the case when the time signal f(f) is real; that 
is, loosely speaking, a consequence of the fact that we have decomposed, or analysed 
f(d, relative to complex exponentials rather than real-valued sines and cosines. The 
second common feature is that all the amplitude spectra decrease rapidly as @ increases. 
This means that most of the information concerning the ‘shape’ of the signal f(t) 
is contained in a fairly small interval of the frequency axis around w= 0. From another 
point of view, we see that a device capable of passing signals of frequencies up to 
about w = 3/T would pass a reasonably accurate version of the rectangular pulse of 
Example 8.3. 


www.2Ofile.org 


8.2.4 Exercises 


8.2 THE FOURIER TRANSFORM 651 


Whenever possible check your answers using MATLAB or MAPLE. 


il Calculate the Fourier transform of the two-sided 
exponential pulse given by 


at (t <0) 


e€ 
f(t) \ > 0) (a > 0) 


2 Determine the Fourier transform of the ‘on-off’ 
pulse shown in Figure 8.11. 


Figure 8.11 The ‘on-off’ pulse. 


3 A triangular pulse is defined by 


(A/T)t + A 
(-A/T)t + A 


_ (-T<t<0) 
A= (0<t<T) 


Sketch f(#) and determine its Fourier transform. 
What is the relationship between this pulse and 
that of Exercise 2? 


4 Determine the Fourier transforms of 


fi) = i (\t| < 2) 


0 ({t| > 2) 
Bes a (\t] < 1) 
0 (lt) >1) 


Sketch the function A(t) = f(4) — g(t) and determine 
its Fourier transform. 


5 Calculate the Fourier transform of the ‘off—on—off’ 
pulse f(7) defined by 


O: fe 
Li). (at ep Sai 
(OSs 1 Cl<r=1j 
“) (1 = peo) 
6. 473) 
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Show that the Fourier transform of 


sinat (|t| < 1/a) 


f(t) -| 
0 (\t] > n/a) 


is 


j2asin(n@/a) 


2 2 
@-a 


Calculate the Fourier transform of 


f(t) =e™ sin Wot H(t) 


Based on (8.10) and (8.11), define the Fourier sine 
transform as 


F(x) = | f(t) sin xt dt 
0 
and the Fourier cosine transform as 


F(x) = | f(t) cos xt dt 


Show that 
0 (t <0) 
f(t) =jcosat (0 <t<a) 
0 (t >a) 


has Fourier cosine transform 


= +x)a 4 aaa 


l+x 


Show that the Fourier sine and cosine transforms of 


0 (t<0) 
f(t) =41 (0<tSa) 
0 (t>a) 
are 
l—cosxa_ sinxa 
x , x 
respectively. 


Find the sine and cosine transforms of 


ft) =e" H(f) (a > 0). 
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Properties of the Fourier transform 


In this section we establish some of the properties of the Fourier transform that allow 
its use as a practical tool in system analysis and design. 


8.3.1 The linearity property 


Linearity is a fundamental property of the Fourier transform, and may be stated 
as follows. 


If f( and g(t) are functions having Fourier transforms F(j@) and G(j@) respect- 
ively, and if @ and f are constants, then 


Fi of(t) + Bg} = CAL SO} + BF{g(} = OF (jo) + BG jo) (8.22) 


As a consequence of this, we say that the Fourier transform operator ¥ is a linear 
operator. The proof of this property follows readily from the definition (8.15), since 


Flocf(t) + Bglt)} | [arf(t) + Bele" de 


o| finer + 6] g(the dt 


= AF(jo@) + BG(jo) 


Clearly the linearity property also applies to the inverse transform operator ¥"'. 


8.3.2 Time-differentiation property 
If the function /(f) has a Fourier transform F(jq@) then, by (8.16), 


_ A. : : jot 
S(O) = al F(j@)e da 


Differentiating with respect to t gives 


ee ee ee rn 1 nee 
v.| alFUme jeo= | (j@)F(jo)e"" do 


oo 


implying that the time signal df/dt is the inverse Fourier transform of (j@)F(jq@). In 
other words 


ee lael a- 
ata = (ja) F(j@) 


Repeating the argument n times, it follows that 
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capa = (j)"FUio) 
dt 


(8.23) 


The result (8.23) is referred to as the time-differentiation property, and may be used 


to obtain frequency-domain representations of differential equations. 


Show that if the time signals y(‘) and u(f) have Fourier transforms Y(j@) and U(ja) 


respectively, and if 
dy) 4 34). ayy) = 394 + o,(4) 
dt dt dt 
then Y(j@) = G(j@)U(j@) for some function G(j@). 


Taking Fourier transforms throughout in (8.24), we have 
2 
+ {ex + 30 5 7910} 7 {saul ‘ 2010} 
dt dt dt 
which, on using the linearity property (8.22), reduces to 
2 
o {fun + ag {eu + TFLy(t)} = ag {dut| + 2Fu(t)} 
dt dt dt 
Then, from (8.23), 
(j@) Y(j@) + 3(j@) ¥( jo) + 7Y(j@) = 3(j@)U(j@) + 2U(ja) 
that is, 
(-@ + j3@+ 7)Y(jo) = (j3@ + 2)U(jo) 
giving 
Y(j@) = G(j@)U(jo) 
where 


: 2+ 43 
G(jo) = =e — 
7-@+)j30@ 


(8.24) 


The reader may at this stage be fearing that we are about to propose yet another 
method for solving differential equations. This is not the idea! Rather, we shall show 
that the Fourier transform provides an essential tool for the analysis (and synthesis) of 


linear systems from the viewpoint of the frequency domain. 


Time-shift property 


If a function /(f) has Fourier transform F'(j@) then what is the Fourier transform of the 


shifted version g(t) = f(t — T), where T is a constant? From the definition (8.15), 
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Example 8.6 


Figure 8.12 
Rectangular pulse 
of Example 8.6. 


Solution 


8.3.4 


co 


F{ g(t) } =| g(the dt =| ft— Dei” at 


—co 


Making the substitution x = t— T, we have 


co 


F{ g(t)} = | f(xyerr"? dx =e" | f(xje??"dx = e!” F( ja) 


—co 


that is, 
Fit — D} =e" F(jo) (8.25) 
The result (8.25) is known as the time-shift property, and implies that delaying a signal 


by a time T causes its Fourier transform to be multiplied by e7”. 
Since 


|e#°"| =|cos ot —jsinwt|=|,/(cos?@t +sin?@r) | = 1 

we have 
|e?" F(jo) | =| Fo) | 

indicating that the amplitude spectrum of /(¢— 7) is identical with that of f(4). However, 
arg[e!°' F(j@)] = arg F(j@) — arge!”’ = arg F(j@) — wt 


indicating that each frequency component is shifted by an amount proportional to its 
frequency @. 


Determine the Fourier transform of the rectangular pulse f(t) shown in Figure 8.12. 


This is just the pulse of Example 8.3 (shown in Figure 8.6), delayed by 7. The pulse of 
Example 8.3 had a Fourier transform 2AT sinc wT, and so, using the shift property 
(8.25) with T= T, we have 


Ff} = F(j@) =e!" 2AT sinc @T = 2AT e sinc oT 


Frequency-shift property 


Suppose that a function /(/) has Fourier transform F'(j@). Then, from the definition 
(8.15), the Fourier transform of the related function g(t) = e’° f(t) is 
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Solution 
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co 


Ff g(t)} = | e' f(t)e dt = | fier? dt 


—oo 


7 | fie” dt, where @ = w@- a 


=F(j@), by definition 
Thus 


Fi el f(t)} = F(j(@— @»)) (8.26) 


The result (8.26) is known as the frequency-shift property, and indicates that multi- 
plication by e’° simply shifts the spectrum of f(t) so that it is centred on the point 
@ = @, in the frequency domain. This phenomenon is the mathematical foundation 
for the process of modulation in communication theory, illustrated in Example 8.7. 


Determine the frequency spectrum of the signal g(t) = f(t) cos @,t. 


ja.t 


tte “c) | it follows, using the linearity property (8.22), that 
Fa} = Az f(y(e" + e+) } 
= APO} + SFL Me} 
If F{ f(t)} = F (jo) then, using (8.26), 
F{ f(t) cos Wt} = F{g(f)} = 3 F(j(@—- @,)) + 5 F((@ + @,)) 


The effect of multiplying the signal /(‘) by the carrier signal cos a, is thus to produce 
a signal whose spectrum consists of two (scaled) versions of F'(j@), the spectrum of 
f(t): one centred on @ = @, and the other on @ = —a,. The carrier signal cos @,¢ is said 
to be modulated by the signal /(?). 


= =, il 
Since cos @,t = 5(€ 


Demodulation is considered in Exercise 5, Section 8.10, and the ideas of modulation 
and demodulation are developed in Section 8.8. 


The symmetry property 


From the definition of the transform pair (8.15) and (8.16) it is apparent that there is 
some symmetry of structure in relation to the variables ¢ and w. We can establish the 
exact form of this symmetry as follows. From (8.16), 


f(t) = | Fae" da 
2m} 
or, equivalently, by changing the ‘dummy’ variable in the integration, 


I= | F(iy)e” dy 
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so that 
2nf(-t) = | : F(jy)e" dy 
or, on replacing f¢ - 
2nf(-@) = | ; F(jy)e?* dy (8.27) 


The right-hand side of (8.27) is simply the definition (8.15) of the Fourier transform 
of F(jf), with the integration variable t replaced by y. We therefore conclude that 


FF (jt)} = 20f(-o) (8.28a) 
given that 
F{f(O} =F Go) (8.28b) 


What (8.28) tells us is that if f(t) and F'(jq@) form a Fourier transform pair then F'(jt) 
and 21f(—@) also form a Fourier transform pair. This property is referred to as the 
symmetry property of Fourier transforms. It is also sometimes referred to as the 
duality property. 


Example 8.8 — Determine the Fourier transform of the signal 
C sin at (t #0) 


g(t) = Csincat = at (8.29) 
G (t= 0) 


Solution From Example 8.3, we know that if 


A (=) 


8.30 
0 (\t|> 7) — 


f= 


then 
Fi f(t)} = F(j@) = 2AT sinc oT 


Thus, by the symmetry property (8.28), F'( jf) and 21 f(—@) are also a Fourier transform 
pair. In this case 


F (jt) = 2AT sinc tT 
and so, choosing 7 = a and A = C/2a to correspond to (8.29), we see that 
F (jt) = C sine at = g(t) 


has Fourier transform 27f/(—@). Rewriting (8.30), we find that, since|@| = |—a|, 


2nC/2a (|@| <a) a (la| < a) 


FC si = = 
ad 0 (aj>a) | 0 (al>a) 
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A graph of g(t) and its Fourier transform G(j@) = 27/(—@) is shown in Figure 8.13. 


Figure 8.13 

The Fourier 
transform pair 
g(t) and G( ja) of 
Example 8.8. 


G(jw) 
aCla 


-a O a w 


2) Using the MATLAB commands 


syme wt ac 
iDP=sc ope Leis (C’ Sina (ee) / (artic) pe, wi) 2 
F=simple(F) 
returns 
F=C*pi* (-Heaviside (w-a) +Heaviside(wta))/a 


which is the answer given in the solution expressed in terms of Heaviside functions. 


8.3.6 Exercises 


Whenever possible check your answers using MATLAB or MAPLE. 


11 Use the linearity property to verify the result in 14 ~— Calculate the Fourier transform of the windowed 
Exercise 4. cosine function 
t) = cos @ot [H(t + } T) - Ht- 3 T 
12 If (A) and u() are signals with Fourier transforms Fi) = cos @ol Ae 5 E)— Hat) 


eed Ce) Tesaschvel ys ene 15 _ Find the Fourier transform of the shifted form of 


2 5 A * 
d vet) + 3du t) +O =u the windowed cosine function 
dt dr 2(#) = cos @ t[H(t) - Hit - T)] 


show that Y(j@) = H(j@)U(j@) for some function 
H(jq@). What is H(j@)? 16 ~— Calculate the Fourier transform of the windowed 
sine function 
13. Use the time-shift property to calculate the Fourier f= sin eft + 1) - HE - D] 


transform of the double pulse defined by 


1 (1 <|t|) <2) 


0 (otherwise) 


f(t) =| 
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PBA The frequency response 


8.4.1 


In this section we first consider the relationship between the Fourier and Laplace transforms, 
and then proceed to consider the frequency response in terms of the Fourier transform. 


Relationship between Fourier and Laplace transforms 


The differences between the Fourier and Laplace transforms are quite subtle. At first 
glance it appears that to obtain the Fourier transform from the Laplace transform we 
merely write j@ for s, and that the difference ends there. This is true in some cases, but 
not in all. Strictly, the Fourier and Laplace transforms are distinct, and neither is a 
generalization of the other. 

Writing down the defining integrals, we have 


The Fourier transform 


Fi f(t)} = | fie" dt (8.31) 


The bilateral Laplace transform 


Lyf f(t)} = | f(tve “dt (8.32) 


The unilateral Laplace transform 


co 


LA f(t)} = | £0 edt (8.33) 


0 


There is an obvious structural similarity between (8.31) and (8.32), while the connec- 
tion with (8.33) is not so clear in view of the lower limit of integration. In the Laplace 
transform definitions recall that s is a complex variable, and may be written as 


(6470 (8.34) 


where o and @ are real variables. We can then interpret (8.31), the Fourier transform of 


f(t), as a special case of (8.32), when o = 0, provided that the Laplace transform exists 


when o = 0, or equivalently when s = jq@ (that is, s describes the imaginary axis in the 
s plane). If we restrict our attention to causal functions, that is functions (or signals) that 
are zero whenever ¢ < 0, the bilateral Laplace transform (8.32) is identical with the 
unilateral Laplace transform (8.33). The Fourier transform can thus be regarded as a 
special case of the unilateral Laplace transform for causal functions, provided again that 
the unilateral Laplace transform exists on the imaginary axis s = ja. 

The next part of the story is concerned with a class of time signals f(t) whose 
Laplace transforms do exist on the imaginary axis s = j@. Recall from (5.71) that a 
causal linear time-invariant system with Laplace transfer function G(s) has an impulse 
response A(t) given by 
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Pole locations for 
G(s) and the region 
of existence of 
G(s). 


Example 8.9 
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Im(s) A 
* * > 
-c?| O Re(s) 
A(t) = L'{G(s)} =g()H(, say (8.35) 


Furthermore, if the system is stable then all the poles of G(s) are in the left half-plane, 
implying that g(t) H(t) > 0 as t > ©. Let the pole locations of G(s) be 


Pi» Prr+ ++ Pr 
where 

Pr= a; + jb, 
in which a,, b, are real and a, # 0 for k = 1, 2,..., n. Examples of such poles are 
illustrated in Figure 8.14, where we have assumed that G(s) is the transfer function of 
a real system so that poles that do not lie on the real axis occur in conjugate pairs. As 


indicated in Section 5.2.3, the Laplace transfer function G(s) will exist in the shaded 
region of Figure 8.14 defined by 


Re(s) > —c? 
where —c’ is the abscissa of convergence and is such that 
0<c’?< mina; 


The important conclusion is that for such systems G(s) always exists on the imaginary 
axis s = j@, and so h(t) = g(t) H(t) always has a Fourier transform. In other words, we 
have demonstrated that the impulse response function A(t) of a stable causal, linear 
time-invariant system always has a Fourier transform. Moreover, we have shown that 
this can be found by evaluating the Laplace transform on the imaginary axis; that is, 
by putting s = j@ in the Laplace transform. We have thus established that Fourier 
transforms exist for a significant class of useful signals; this knowledge will be used 
in Section 8.4.2. 


Which of the following causal time-invariant systems have impulse responses that 
possess Fourier transforms? Find the latter when they exist. 


(a) foto " ao + 2y(t) = u(t) 
t t 


(b) oto + Gy(t) = u(t) 
, 


(c) 510) 4 WO 6 5G) = out) + HO 
dt dt dt 
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Solution 


8.4.2 


Assuming that the systems are initially in a quiescent state when ¢ < 0, taking Laplace 
transforms gives 


(a) ¥(s)= + ~(s) = G(s) U(s) 
s +3s4+2 

(b) Ys) = G(s) = Gxls)U(s) 
s +@ 


(c) ¥(s) = S44 Ws) = G,(s)U(s) 
s+ts4+1 


In case (a) the poles of G,(s) are at s=—1 and s =—2, so the system is stable and the 
impulse response has a Fourier transform given by 


: 1 1 
C10) = es) = 


8 +3st¢2|_ 2-@+j3o 


JO 
__2-@ -j30 _ _ (2-@) -j3o 
(2-W) +90 @w+50 +4 

In case (b) we find that the poles of G,(s) are at s = j@ and s = —jq@; that is, on the 


imaginary axis. The system is not stable (notice that the impulse response does not 


decay to zero), and the impulse response does not possess a Fourier transform. 


In case (c) the poles of G,(s) are at s = —} + j},/3 and s= —} — j}\3. Since these are 


in the left half-plane, Re(s) < 0, we conclude that the system is stable. The Fourier 
transform of the impulse response is then 


Gin=— 
l1-@+ja 


The frequency response 


For a linear time-invariant system, initially in a quiescent state, having a Laplace transfer 
function G(s), the response y(f) to an input u(f) is given in (5.66) as 


Y(s) = G(s)U(s) (8.36) 


where Y(s) and U(s) are the Laplace transforms of y(t) and u(t) respectively. In 
Section 5.8 we saw that, subject to the system being stable, the steady-state response 
y(t) to a sinusoidal input u(t) = A sin wt is given by (5.101) as 


Vest) = A|G(jo)| sin[@t + arg G(jo)] (8.37) 


That is, the steady-state response is also sinusoidal, with the same frequency as the 
input signal but having an amplitude gain |G(jq@)| and a phase shift arg G(ja@). 
More generally, we could have taken the input to be the complex sinusoidal signal 


u(t) =A ei” 
and, subject to the stability requirement, showed that the steady-state response is 

y(t) = AG(jo) e'”" (8.38) 
or 


Vogt) = A|G(ja)| lores G00 (8.39) 


www.2Ofile.org 


Example 8.10 


Solution 
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As before, |G(j@)| and argG(jq) are called the amplitude gain and phase shift 
respectively. Both are functions of the real frequency variable @, and their plots versus 
@ constitute the system frequency response, which, as we saw in Section 5.8, charac- 
terizes the behaviour of the system. Note that taking imaginary parts throughout in (8.39) 
leads to the sinusoidal response (8.37). 

We note that the steady-state response (8.38) is simply the input signal Ae!” multi- 
plied by the Fourier transform G(jq@) of the system’s impulse response. Consequently 
G(jq) is called the frequency transfer function of the system. Therefore if the system 
represented in (8.36) is stable, so that G(j@) exists as the Fourier transform of its 
impulse response, and the input u(t) = £'{ U(s)} has a Fourier transform U(jq@), then 
we may represent the system in terms of the frequency transfer function as 


Y(j@) = G(j@)U(jo) (8.40) 


Equation (8.40) thus determines the Fourier transform of the system output, and can 
be used to determine the frequency spectrum of the output from that of the input. This 
means that both the amplitude and phase spectra of the output are available, since 


|¥(j@)| =|G(j@)| |Uj@)| (8.41a) 
arg Y(j@) = arg G(jq@) + arg U( jo) (8.41b) 


We shall now consider an example that will draw together both these and some earlier ideas 
which serve to illustrate the relevance of this material in the communications industry. 


A signal f(4) consists of two components: 


(a) asymmetric rectangular pulse of duration 27 (see Example 8.3) and 


(b) asecond pulse, also of duration 27 (that is, a copy of (a)), modulating a signal with 
carrier frequency @, = 3 (the process of modulation was introduced in Section 8.3.4). 


Write down an expression for f(t) and illustrate its amplitude spectrum. Describe the 
amplitude spectrum of the output signal if f(A) is applied to a stable causal system with 
a Laplace transfer function 


1 


= 
s+ 2s+1 


Denoting the pulse of Example 8.3, with T= 7, by P,(t), and noting the use of the term 
‘carrier signal’ in Example 8.7, we have 


S(O = PD) + (cos 30) P, (0) 
From Example 8.3, 
F{P_(t)} =2n sinc on 
so, using the result of Example 8.7, we have 
Fi f(t)} = F (jo) = 20 sinc wn + } [20 sinc(@— 3)n + 20 sinc(@ + 3)r] 


The corresponding amplitude spectrum obtained by plotting |F(jq@)| versus @ is illus- 
trated in Figure 8.15. 
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Figure 8.15 | F(jea) | 
Amplitude spectrum 
of the signal 

P(t) + (cos 3t)P_(t). 


2n 


=6 Had-F 221% [ 2 5 4s 6 os 
Since the system with transfer function 


Gis) == 
s + \2s+ 1 

is stable and causal, it has a frequency transfer function 
| 

1- @ + jj2@ 

so that its amplitude gain is 


|G(jo) | = —— 
\(@ +1) 


G(j@) = 


The amplitude spectrum of the output signal | Y(j@)| when the input is f(f) is then 
obtained from (8.41a) as the product of | F(j@)| and |G(j@)|. Plots of both the amplitude 
gain spectrum |G(jq@)| and the output amplitude spectrum | Y(j@)| are shown in Figures 
8.16(a) and (b) respectively. Note from Figure 8.16(b) that we have a reasonably good 
copy of the amplitude spectrum of P,(f) (see Figure 8.9 with A = 7, T= 1). However, 
the second element of f(t) has effectively vanished. Our system has ‘filtered out’ this 
latter component while ‘passing’ an almost intact version of the first. Examination of 
the time-domain response would show that the first component does in fact experience 
some ‘smoothing’, which, roughly speaking, consists of rounding of the sharp edges. 
The system considered here is a second-order ‘low-pass’ Butterworth filter (introduced 
in Section 6.10.1). 


lY¥(jw)| 
2K 


6-5 -4-3-2-19 1} 23456 @ 6-5 4-32-19 123456 # 
(a) (b) 


Figure 8.16 (a) Amplitude gain spectrum of the system with G(s) = 1/(s? + /2.s + 1); (b) amplitude spectrum of the output 
signal | Y(j@)| of Example 8.10. 
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8.4.3 Exercises 


Find the impulse response of systems (a) and (c) 
of Example 8.9. Calculate the Fourier transform 

of each using the definition (8.15), and verify the 
results given in Example 8.9. 


Use the time-shift property to calculate the Fourier 
transform of the double rectangular pulse f(A) 
illustrated in Figure 8.17. 


nite) 
A 


t + > 
-1-T -t -t4+T oO tT-T «t t+T ? 


Figure 8.17 The double rectangular pulse of 
Exercise 18. 


The system with transfer function 


1 
CQ) Ser 
s+ /2s+1 


was discussed in Example 8.10. Make a 
transformation 


20 


21 


663 


pal 
Ss 
and write down G(s’). Examine the frequency 
response of a system with transfer function G(s’) 
and in particular find the amplitude response 
when @= 0 and as @ > oe. How would you 
describe such a system? 


Use the symmetry property, and the result of 
Exercise 1, to calculate the Fourier transform of 


f= 


att 


Sketch f(#) and its transform (which is real). 


Using the results of Examples 8.3 and 8.7, calculate 

the Fourier transform of the pulse-modulated signal 
S(t) = Pr (t) cos Wot 

where 


1 (t]}<T) 


ran=| 
0 (|t| > T) 


is the pulse of duration 27. 


Transforms of the step and impulse functions 


In this section we consider the application of Fourier transforms to the concepts of 
energy, power and convolution. In so doing, we shall introduce the Fourier transform 
of the Heaviside unit step function H(t) and the impulse function 6(?). 


8.5.1 Energy and power 


In Section 7.6.4 we introduced the concept of the power spectrum of a periodic signal 

and found that it enabled us to deduce useful information relating to the latter. In this 

section we define two quantities associated with time signals /(4), defined for —co < t < 9, 

namely signal energy and signal power. Not only are these important quantities in them- 

selves, but, as we shall see, they play an important role in characterizing signal types. 
The total energy associated with the signal f(f) is defined as 


Ee | [f(y de 
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Example 8.11 


Solution 


If f(t) has a Fourier transform F'(jq@), so that, from (8.16), 
fi) == | F(ja) edo 
21 ” 
then (8.42) may be expressed as 


z= | foros | no) | Powe 


—co 


On changing the order of integration, this becomes 


ae x | 7 rj0)| | fit) ea do (8.43) 


From the defining integral (8.15) for F(j@), we recognize the part of the integrand 
within the square brackets as F(—j@), which, if f(t) is real, is such that F(-j@) = 
F’*(jq@), where F'*(jq@) is the complex conjugate of F'(j@). Thus (8.43) becomes 


E= +| F(jo)F*(jo) do 


so that 


co 


| [AV dt = | | F(ja) F dw (8.44) 


Equation (8.44) relates the total energy of the signal f(/) to the integral over all fre- 
quencies of |F(jq@)|’. For this reason, |F'(j@)|’ is called the energy spectral density, 
and a plot of |F(j@)|’ versus @ is called the energy spectrum of the signal f(#). The 
result (8.44) is called Parseval’s theorem, and is an extension of the result contained 
in Theorem 7.6 for periodic signals. 


Determine the energy spectral densities of 


(a) the one-sided exponential function f(t) = e“ H(A) (a > 0), 
(b) the rectangular pulse of Figure 8.6. 


(a) From (8.17), the Fourier transform of f(f) is 


F(jo) = 42 
@ 


The energy spectral density of the function is therefore 


. . . —j@ a+jo 
|F(jo) ) = F(ja)F*(jo) = $18 4410 
at+@Ma+@ 


that is, 
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1 


2 2 
a+@ 


|F (jo) = 


(b) From Example 8.3, the Fourier transform F'(jq@) of the rectangular pulse is 
F(j@) = 2AT sinc oT 
Thus the energy spectral density of the pulse is 


|F (jo)? =44°7° sine’ oT 


There are important signals f(t), defined in general for —co < ¢ < oe, for which the 
integral f°. [f(A] dt in (8.42) either is unbounded (that is, it becomes infinite) or does 
not converge to a finite limit; for example, sin ¢. For such signals, instead of considering 
energy, we consider the average power P, frequently referred to as the power of the 
signal. This is defined by 


HP 
j= hime | [ft] dt (8.45) 
TH Ti 


=I 


Note that for signals that satisfy the Dirichlet conditions (Theorem 8.1) the integral 
in (8.42) exists and, since in (8.45) we divide by the signal duration, it follows that 
such signals have zero power associated with them. 

We now pose the question: ‘Are there other signals which possess Fourier transforms?’ 
As you may expect, the answer is ‘Yes’, although the manner of obtaining the transforms 
will be different from our procedure so far. We shall see that the transforms so obtained, 
on using the inversion integral (8.16), yield some very ‘ordinary’ signals so far excluded 
from our discussion. 

We begin by considering the Fourier transform of the generalized function 6(A), the 
Dirac delta function introduced in Section 5.5.8. Recall from (5.49) that d(t) satisfies 
the sifting property; that is, for a continuous function g(t), 


) g(t)6(t — c) dt = es @<¢< B) 


0 otherwise 


a 


Using the defining integral (8.15), we readily obtain the following two Fourier 
transforms: 


Fi 5(t)}= | d(t)e dt =1 (8.46) 


FL S(t — t)} = | 5(t — te dt = e 1 (8.47) 


These two transforms are, by now, unremarkable, and, noting that | e! “| = 1, we illustrate 
the signals and their spectra in Figure 8.18. 
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Figure 8.18 

(a) O(f) and its 
amplitude spectrum; 
(b) d(t — t)) and its 
amplitude spectrum. 


O(t) IF (jea)l 
—_— 
O t oO w 
(a) 
OU ~ tg) IF(jwil 
—s 
O t t Oo w 
(b) 


These results may be confirmed in MATLAB. Using the commands 


syms w t 
D=eyam (7 Daieere (te) ? )) 9 
F=fourier(D,t,w) 


returns 


oe) 
in agreement with (8.46); whilst the commands 
syms wt T 
Dil =syam ( Dizae (ew) 7) ¢ 
Fl=fourier(D1,t,w) 
return 
Fl=exp (-i*T*w) 
which confirms (8.47), with T replacing 4. 
Likewise in MAPLE the commands 


with(inttrans) : 
fourier (Dirac(t),t,w); 


return the answer 1. 


We now depart from the definition of the Fourier transform given in (8.15) and seek 
new transform pairs based on (8.46) and (8.47). Using the symmetry (duality) property 
of Section 8.3.5, we deduce from (8.46) that 


1 and 2n26(—@) = 2n6(@) (8.48) 
is another Fourier transform pair. Likewise, from (8.47), we deduce that 
e* and = 2n6(-@— ty) 
is also a Fourier transform pair. Substituting 4) = —@, into the latter, we have 
and = 2n6(@,) — @) = 205(@— @) (8.49) 


as another Fourier transform pair. 

We are thus claiming that in (8.48) and (8.49) that f,(f) = 1 and f(t) = e'°", which 
do not have ‘ordinary’ Fourier transforms as defined by (8.15), actually do have 
‘generalized’ Fourier transforms given by 


F (jo) = 276(o) (8.50) 
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F,(j@) = 2n6(@— @,) (8.51) 


respectively. 

The term ‘generalized’ has been used because the two transforms contain the gener- 
alized functions 6(@) and 6(@ — @,). Let us now test our conjecture that (8.50) and (8.51) 
are Fourier transforms of /,() and f,(4) respectively. If (8.50) and (8.51) really are Fourier 
transforms then their time-domain images f,(f) and f,(f) respectively should reappear via 
the inverse transform (8.16). Substituting F',(jq@) from (8.50) into (8.16), we have 


enon = rGoe*so= 2] 2n6()e dw = 1 


so f,(f) = 1 is recovered. 
Similarly, using (8.51), we have 


F" {F,(j@)} = +| 2nd(@- a,c" dw= ee!" 


so that f(t) = e’°” is also recovered. 


Our approach has therefore been successful, and we do indeed have a way of gener- 
ating new pairs of transforms. We shall therefore use the approach to find generalized 
Fourier transforms for the signals 


A =cosQ@ ft, f(t) = sin Wot 
Since 
f(t) = cos Wot = ea +e 1%) 
the linearity property (8.22) gives 
Fl fy(t)} = LF Lely + 1FL eM} 
which, on using (8.49), leads to the generalized Fourier transform pair 
F{cos Wot} = T[d(@— yo) + 6(@+ Wy)| (8.52) 
Likewise, we deduce the generalized Fourier transform pair 
F{sin Wot} = jn[6(@+ @)) — 6(@— @)] (8.53) 


The development of (8.53) and the verification that both (8.52) and (8.53) invert 
correctly using the inverse transform (8.16) is left as an exercise for the reader. 

It is worth noting at this stage that defining the Fourier transform ¥{ f(t)} of f(4) 
in (8.15) as 


Fi f(t} = | f(tye’ dt 


whenever the integral exists does not preclude the existence of other Fourier transforms, 
such as the generalized one just introduced, defined by other means. 
It is clear that the total energy 


E =| cos Wot dt 
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Example 8.12 


Solution 


Example 8.13 


Solution 


associated with the signal f,(t) = cos @of is unbounded. However, from (8.45), we can 
calculate the power associated with the signal as 


T/2 T/2 
P=limt cos Wot dt = lim a oie tain 2Wot =s 
To0 To T 206 


-1/2 -7/2 


Thus, while the signal f(t) = cos@pt has unbounded energy associated with it, its 
power content is }. Signals whose associated energy is finite, for example f(t) = e“H(t) 
(a > 0), are sometimes called energy signals, while those whose associated energy is 
unbounded but whose total power is finite are known as power signals. The concepts 
of power signals and power spectral density are important in the analysis of random 
signals, and the interested reader should consult specialized texts. 


Suppose that a periodic function f(f), defined on —co < t < oo, may be expanded in a 
Fourier series having exponential form 


fj=> Be 


n=—co 


What is the (generalized) Fourier transform of f(t)? 


From the definition, 


F{ f(t) } _ + 3 F, i _ 3 F, Fie"? 


n=—co n=—co 


which, on using (8.49), gives 
F{f)} = Y F,276(@ — na) 


n=—0o 


That is, 


F{)} = 2m) F,5(w— ne) 


n=—co 


where F,, (—co <n < ce) are the coefficients of the exponential form of the Fourier series 
representation of f(7). 


Use the result of Example 8.12 to verify the Fourier transform of f(4) = cos @ of given in (8.52). 


Since 


j@ot —j@ot 


f= cos agt=je'" + fe 
the F, of Example 8.12 are 
F,=F,=}5 


F,=0 (n##l) 
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Thus, using the result 
F{ fit)} = 2nY" F,6(@ — a) 
we have 
F{Cos Wot} = 20F_,6(@ + @)) + 20F,6(@— Wy) 
=T[6(@+ @,) + 6(@—- @)] 


in agreement with (8.52). 


&) Confirm this answer using the MATLAB commands 


Syme wok ia 
Sr ouneieie (COS (Ae) pic pW) 


where a has been used to represent @». 


Example 8.14 Determine the (generalized) Fourier transform of the periodic ‘sawtooth’ function, 
defined by 


fo=% (0 <1<2T7) 


f(t + 2T) =f) 


Solution In Example 7.19 we saw that the exponential form of the Fourier series representation 


of f(f) is 


f= ¥ Fe" 


n=—' 


with 


It follows from Example 8.12 that the Fourier transform #{ f(t)} is 


Ff} = Fj@) = 4nd(w) + Y j45(e ~ nev) 


n#0 


= 4n5(@) + j4 y 5( 0 z a) 
n 


n=—0co 


n#0 
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Example 8.15 


Figure 8.19 
Unit impulse train 


S(t) = Lye. Ot — nT). 


Solution 


Thus we see that the amplitude spectrum simply consists of pulses located at integer 
multiples of the fundamental frequency @, = 71/T. The discrete line spectra obtained via 
the exponential form of the Fourier series for this periodic function is thus reproduced, 
now with a scaling factor of 27. 


Determine the (generalized) Fourier transform of the unit impulse train f(f) = %1,----. 6(t— nT) 
shown symbolically in Figure 8.19. 


f(y 
1 


O T 27 37 47 5ST t 


Although /(f) is a generalized function, and not a function in the ordinary sense, it 
follows that since 


f(t + kT) = x O(t+(k-—n)T) (Kk an integer) 


n=—0o 


¥ Or - mT) (m=n-—k) 


m=—oco 


=f 
it is periodic, with period 7. Moreover, we can formally expand /(f) as a Fourier series 


— jnot 27 
fo= > me (w= 2B) 


with 


T/2 T/2 : 1 
-jno -jn@ot 
Py = A{ f(pje’ var=4| 5(t)e- : dt= 5, for all n 


-T/2 
It follows from Example 8.12 that 
a Es 
Fl fii)}= on poo — NW») = oD PO — NW») 
Thus we have shown that 


+ ¥ Ht = nt)| = oy 5(@ — n@p) (8.54) 


n=—0o n=—0o 


where @, = 21/T. That is, the time-domain impulse train has another impulse train as 
its transform. We shall see in Section 8.6.4 that this result is of particular importance in 
dealing with sampled time signals. 
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Following our successful hunt for generalized Fourier transforms, we are led to con- 
sider the possibility that the Heaviside unit step function H(t) defined in Section 5.5.1 
may have a transform in this sense. Recall from (5.56) that if 


S() = HO) 
then 
df(t) _ 
dt on) 


From the time-differentiation property (8.23), we might expect that if 
F{H()} = H(ja) 
then 
(j@)H(jo) = F{d(t)} = 1 (8.55) 


Equation (8.55) suggests that a candidate for H(j@) might be 1/j@, but this is not the 
case, since inversion using (8.16) does not give H(t) back. Using (8.16) and complex 
variable techniques, it can be shown that 


ao) i (¢>0) 

gi| 1 1 a 1 

F jo Ge a 0 (t=0) = ;sgn(f) 
-_ -} (t<0) 


where sgn(f) is the signum function, defined by 


1 (t>0) 
sen(t)=5 0 (t=0) 
-l (t<0) 


(Note: This last result may be obtained in terms of Heaviside functions using the 
MATLAB commands 


syms w t 
(Pao ie@ulieaLene (AN // (Lp) )} 


or using the MAPLE commands 


with(inttrans) : 
invfourier (1(I*w) ,w,t); 


However, we note that (8.55) is also satisfied by 
Ajo) = + + cd() (8.56) 
jo 


where c is a constant. This follows from the equivalence property (see Definition 5.2, 
Section 5.5.11) f(@)6(@) = f(0)d(@) with f(@) = ja, which gives 


(j@)H(jo) = 1 + (j@)c5(@) = 1 
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EY 


Inverting (8.56) using (8.16), we have 


_gtjL = A. : i jot 
gth=F is + <3.| al E + <3(0) e da 
c/2n +} (t>0) 
= 4 c/2n (t=0) 
c/2n-} (t <0) 


and, choosing c = 7, we have 


1 (t>0) 
g(t) =) 3 (t=0) 
0 (t<0) 


Thus we have (almost) recovered the step function H(#). Here g(t) takes the value } at 
t = 0, but this is not surprising in view of the convergence of the Fourier integral at 
points of discontinuity as given in Theorem 8.1. With this proviso, we have shown that 


H(jo) = F{H(1)} = Fe + 26(@) (8.57) 


We must confess to having made an informed guess as to what additional term to add in 
(8.56) to produce the Fourier transform (8.57). We could instead have chosen cd(kq) 
with ka constant as an additional term. While it is possible to show that this would not lead 
to a different result, proving uniqueness is not trivial and is beyond the scope of this book. 


Using the MATLAB commands 


syms w t 
H=sym(‘Heaviside(t)'’); 
F=fourier (h,t,w) 


returns 


F=pi*Dirac (w) -i/w 
which, noting that —7 = 1/7, confirms result (8.57). 
The same result is obtained in MAPLE using the commands. 
with(inttrans) : 
fourier (Heaviside(t),t,w); 
Likewise the MATLAB commands 


Syms wo tT 
H=sym( ‘Heaviside (t-T)’); 
F=fourier (H,t,w) 


return 


F=exp (-1*T*w) * (pi*Dirac (w) -i/w) 
which gives us another Fourier transform 
FLA(t — T)} = €3°"(n6(@) + 1/jo) 
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Convolution 


In Section 5.6.6 we saw that the convolution integral, in conjunction with the Laplace 
transform, provided a useful tool for discussing the nature of the solution of a differ- 
ential equation, although it was not perhaps the most efficient way of evaluating the 
solution to a particular problem. As the reader may now have come to expect, in view 
of the duality between time and frequency domains, there are two convolution results 
involving the Fourier transform. 


Convolution in time 


Suppose that 


F{u(t)} = U(jo) =| u(t)e!” dt 


F{v(t)} = V( ja) -| v(the dt 


then the Fourier transform of the convolution 


y(t) = | u(T)v(t — T) dt = u(t) * v(t) (8.58) 


F{y(t)} = Y(jo) = | | u(T)u(t — nar dt 


= | u(T) | ent — say dt 


Introducing the change of variables z > t — tT, T— T and following the procedure for 
change of variable from Section 5.6.6, the transform can be expressed as 


Y(jo) = | u(T) | Haye a a 


- | u(t)e!”" ae] v(z)e!” dz 


so that 

Y(j@) = Uja)V(jo) (8.59) 
That is, 

Flu(t) *v(t)} = F{v(O * uO} = Uja)V(jo) (8.60) 


indicating that a convolution in the time domain is transformed into a product in the 
frequency domain. 
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Convolution in frequency 


If 


oy 


Flu(t)} = U(jo), with w= 2 U(ja) eda 


Fi{v(t)} = Vijo), with v(t) = al V(jo)e'do 
then the inverse transform of the convolution 

U(j@) * V(j@) = f U(jy) V(j(@ — y)) dy 
is given by 


r'ugo)«raor=£| all KGNV te - yar] do 


es eer | ere eS 
= a viv] | Vij(@—y))e vo dy 


A change of variable z — @- y, @ > @ leads to 


F'vgmeroo= 2] wun] | rane"as| a 


L| uunneay| V(jz) e dz 


~ OT 
= 2nu(tu(t) 
That is, 
Flu(Qv(} = 5 Uj) * Vo) (8.61) 


and thus multiplication in the time domain corresponds to convolution in the frequency 
domain (subject to the scaling factor 1/(27)). 


Example 8.16 Suppose that f(t) has a Fourier transform F'(jq@). Find an expression for the Fourier 
transform of g(t), where 


g(t) = | S(t) dt 
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Solution Since 


He-9=|{ 


we can write 


1 (ts?) 
0 (t>Db) 


675 


g(t) =| S(t)H (t — 7) dt = f(t) * H(t) 


the convolution of g(f) and H(t). Then, using (8.60), 


Fig(t)} = G(jo) = F(jo)H(jo) 


which, on using the expression for H(j@) from (8.57), gives 


G(ja) = ae + nF (jo) 5(@) 


so that 


G(jo) = Sue + nF (0)5(o) 


8.5.3 Exercises 


Verify that ¥'{n[6(w@— wy) + 6(@+ @))]} 
= COS Wot. 


Show that A{sin @ot} =jn[6(@+ @)) — 6(@— @,)]- 
Use (8.16) to verify that 

F'§jnx[6(@+ @,) — 6(@— @,)]} = sin Wot 
Suppose that f(7) and g(t) have Fourier transforms 
F(jq) and G(jq@) respectively, defined in the 


‘ordinary’ sense (that is, using (8.15)), and 
show that 


| S(t)G(Gjt) dt = | F(jt)g(t) dt 
This result is known as Parseval’s formula. 


Use the results of Exercise 24 and the symmetry 
property to show that 


(8.62) 


| roxoar= Bl F(j@)G(-jo) do 


26 — Use the convolution result in the frequency domain 
to obtain F{H(t) sin@yt}. 


ZT Calculate the exponential form of the Fourier series 
for the periodic pulse train shown in Figure 8.20. 
Hence show that 


FE f(t)} = aaady. sinc 


n=—0o 


(Joo — NW») 
(@, = 2n/T), and A is the height of the pulse. 


uO) 
<dd> 


SEE 


2T 3F 


> 
t 


Figure 8.20 Periodic pulse train of Exercise 27. 
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PBB The Fourier transform in discrete time 


8.6.1 


8.6.2 


Introduction 


The earlier sections of this chapter have discussed the Fourier transform of signals 
defined as functions of the continuous-time variable t. We have seen that a major area 
of application is in the analysis of signals in the frequency domain, leading to the con- 
cept of the frequency response of a linear system. In Chapter 7 we considered signals 
defined at discrete-time instants, together with linear systems modelled by difference 
equations. There we found that in system analysis the z transform plays a role similar 
to that of the Laplace transform for continuous-time systems. We now attempt to 
develop a theory of Fourier analysis to complement that for continuous-time systems, 
and then consider the problem of estimating the continuous-time Fourier transform in 
a form suitable for computer execution. 


A Fourier transform for sequences 


First we return to our work on Fourier series and write down the exponential form of 
the Fourier series representation for the periodic function F(e!”) of period 2x. Writing 
6= ot, we infer from (7.57) and (7.61) that 


F(e’’) = pes en (8.63) 
where 
f= = | F(e®)e"? do (8.64) 


Thus the operation has generated a sequence of numbers { f,} from the periodic func- 
tion F(e!”) of the continuous variable @. Let us reverse the process and imagine that we 
start with a sequence {g,} and use (8.63) to define a periodic function G’(e'”) such that 


Gel’) = Y & ee (8.65) 


n=—00 


We have thus defined a transformation from the sequence {g,} to G’(e*’). This trans- 
formation can be inverted, since, from (8.64), 


_ | Ge") oe a0 (8.66) 


1 


and we recover the terms of the sequence {g,} from G’(e"*). 
It is convenient for our later work if we modify the definition slightly, defining the 
Fourier transform of the sequence {g,} as 


F{g,} = Ge") = x See (8.67) 


n=—co 
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Solution 


Figure 8.21 
Transform of 
the sequence of 
Example 8.17. 
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whenever the series converges. The inverse transform is then given from (8.66), by 


ees | G(e*) ei 46 (8.68) 
2m) _, 

The results (8.67) and (8.68) thus constitute the Fourier transform pair for the sequence 

{g,}. Note that G(e!”) is a function of the continuous variable 6, and since it is a func- 

tion of e!® it is periodic (with a period of at most 27), irrespective of whether or not the 

sequence {g,} is periodic. 

Note that we have adopted the notation G(e'”) rather than G(@) for the Fourier transform, 
similar to our use of F(jq@) rather than F(q@) in the case of continuous-time signals. In 
the present case we shall be concerned with the relationship with the z transform of 
Chapter 6, where z = r e””, and the significance of our choice will soon emerge. 


Find the transform of the sequence {g;,}"_, where g) =2, g3=g5=l1andg,=g,=0 
fork #0 or 2. 


From the definition (8.67), 


Fig} = Ge*)= ¥ g,e7" 


=g, e+ elt qe =0%9424+e 
= 2(1 + cos 20) = 4cos’0 


In this particular case the transform is periodic of period m, rather than 27. This is 
because g, = g_, = 0, so that cos @ does not appear in the transform. Since G(e!”) is 
purely real, we may plot the transform as in Figure 8.21. 


Gel?) = 4 cos?6A 


Having defined a Fourier transform for sequences, we now wish to link it to 
the frequency response of discrete-time systems. In Section 8.4.2 the link between 
frequency responses and the Fourier transforms of continuous-time systems was estab- 
lished using the Laplace transform. We suspect therefore that the z transform should 
yield the necessary link for discrete-time systems. Indeed, the argument follows closely 
that of Section 8.4.2. 
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For a causal linear time-invariant discrete-time system with z transfer function G(z) 
the relationship between the input sequence {u,} and output sequence { y,} in the trans- 
form domain is given from Section 6.6.1 by 


Y(z) = GU) (8.69) 


where U(z) = F{u,} and Y(z) = Z{y;,}. 
To investigate the system frequency response, we seek the output sequence corre- 
sponding to an input sequence 


{u,} = {Aei*T) = {Aci O= wT (8.70) 


which represents samples drawn, at equal intervals 7, from the continuous-time com- 
plex sinusoidal signal e'. 

The frequency response of the discrete-time system is then its steady-state response 
to the sequence {u,} given in (8.70). As for the continuous-time case (Section 8.4.2), 
the complex form e! is used in order to simplify the algebra, and the steady-state 
sinusoidal response is easily recovered by taking imaginary parts, if necessary. 

From Figure 6.3, we saw that 


F{Aei*} = F{A(e)k} = AZ 


j0 
z—e! 


so, from (8.69), the response of the system to the input sequence (8.70) is determined by 


yj-6p—= (8.71) 
7 

Taking the system to be of order n, and under the assumption that the 7 poles p, 

(r= 1, 2,...,n) of G(z) are distinct and none is equal to e!’, we can expand Y(z)/z in 

terms of partial fractions to give 


YZ) _ __c¢ — »y Cc, (8.72) 
Zz Fs el? Z— Pp, 
where, in general, the constants c, (r= 1, 2,...,) are complex. Taking inverse z trans- 


forms throughout in (8.72) then gives the response sequence as 


(be BH) =) A ah * a ped 


r=1 


that is, 
{yah = cle} +S cfp} (8.73) 
F=1 


If the transfer function G(z) corresponds to a stable discrete-time system then all its 
poles p, (r= 1, 2,..., 7) lie within the unit circle |z| < 1, so that all the terms under the 
summation sign in 1 (8. 7 tend to zero as k — o, This is clearly seen by expressing 
p, in the form p,=|p,|e’ * and noting that if |p,| < 1 then |p,|‘ > 0 as k > . Con- 
sequently, for stable systems the steady-state response corresponding to (8.73) is 


Oe 


Using the ‘cover-up’ rule for partial fractions, the constant c is readily determined from 
(8.71) as 
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Figure 8.22 
Discrete-time system 
of Example 8.18. 
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c=AG(e!”) 
so that the steady-state response becomes 
{¥4,} = AG(e* ){e#*} (8.74) 


We have assumed that the poles of G(z) are distinct in order to simplify the algebra. 
Extending the development to accommodate multiple poles is readily accomplished, 
leading to the same steady-state response as given in (8.74). 

The result (8.74) corresponds to (8.38) for continuous-time systems, and indicates 
that the steady-state response sequence is simply the input sequence with each term 
multiplied by G(e!”). Consequently G(e!”) is called the frequency transfer function of 
the discrete-time system and, as for the continuous case, it characterizes the system’s 
frequency response. Clearly G(e!”) is simply G(z), the z transfer function, with z = e’®, 
and so we are simply evaluating the z transfer function around the unit circle |z| = 1. 
The z transfer function G(z) will exist on |z| = 1 if and only if the system is stable, 
and thus the result is the exact analogue of the result for continuous-time systems in 
Section 8.4.2, where the Laplace transfer function was evaluated along the imaginary 
axis to yield the frequency response of a stable linear continuous-time system. 

To complete the analogy with continuous-time systems, we need one further result. 
From Section 6.6.2, the impulse response of the linear causal discrete-time system with 
z transfer function G(z) is 


Wit = E'{G(z)} = {gihico, say 


Taking inverse transforms then gives 


G2) = 3 az = ¥ ez 
k=0 k=—00 


since g, = 0 (k < 0) for a causal system. Thus 


Ge%)=>¥ ge" 
k=—00 
and we conclude from (8.67) that G(e!”) is simply the Fourier transform of the sequence 
{g,}. Therefore the discrete-time frequency transfer function G(e"”) is the Fourier trans- 
form of the impulse response sequence. 


Determine the frequency transfer function of the causal discrete-time system shown in 
Figure 8.22 and plot its amplitude spectrum. 
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Solution 


Figure 8.23 


Amplitude spectrum 
of the system of 


Example 8.18. 


8.6.3 


Using the methods of Section 6.6.1, we readily obtain the z transfer function as 


2z+1 
z +0.75z + 0.125 


Next we check for system stability. Since z*+ 0.75z + 0.125 = (z + 0.5)(z + 0.25), the 
poles of G(z) are at p, = —0.5 and p, = —0.25, and since both are inside the unit circle 
|z| = 1, the system is stable. The frequency transfer function may then be obtained as 
G(e!*), where 


G(z) = 


jo 
G(e”) = = 2 e ~ 1 
e +0.75e” + 0.125 


To determine the amplitude spectrum, we evaluate | G(e!”)| as 


P jo 
| Gce*)| = 2e +1 


| ei? + 0.75e” + 0.125 


~_ (5 + 4cos6) 
(1.578 + 1.688 cos 6 + 0.25 cos 26) 


A plot of |G(e!”)| versus @ then leads to the amplitude spectrum of Figure 8.23. 


IG(ei9)I 


In Example 8.18 we note the periodic behaviour of the amplitude spectrum, which 
is inescapable when discrete-time signals and systems are concerned. Note, however, 
that the periodicity is in the variable @ = wT and that we may have control over the 
choice of 7, the time between samples of our input signal. 


The discrete Fourier transform 


The Fourier transform of sequences discussed in Section 8.6.2 transforms a sequence 
{g,} into a continuous function G(e!”) of a frequency variable 6, where 9 = wT and T 
is the time between signal samples. In this section, with an eye to computer require- 
ments, we look at the implications of sampling G(e!*). The overall operation will have 
commenced with samples of a time signal {g,} and proceeded via a Fourier transforma- 
tion process, finally producing a sequence {G,} of samples drawn from the frequency- 
domain image G(e!”) of {g,}. 

Suppose that we have a sequence {g,} of N samples drawn from a continuous-time 
signal g(f), at equal intervals 7; that is, 


{gc} = {g(kT) io 
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Using (8.67), the Fourier transform of this sequence is 


Fig} =Ge")= ¥ ge (8.75) 


n=—0o 


where g,=0 (k ¢ [0, N—- 1]). Then, with @= @7, we may write (8.75) as 


N-1 
G(ei”") = by En eer (8.76) 


n=0 


We now sample this transform G(e!®’) at intervals A@ in such a way as to create 
N samples spread equally over the interval 0 < 0 < 27; that is, over one period of the 
essentially periodic function G(e!*). We then have 

NAO=2t 


where A@ is the normalized frequency spacing. Since 9 = wT and T is a constant such 
that AO = T Aa, we deduce that 


hoset (8.77) 
NT 
Sampling (8.76) at intervals A@ produces the sequence 
N-1 ; 
{Gi}ico, where G,= }'g,e0Ne" (8.78) 


n=0 


Since 


(e: _ py g gine NAE 
k+N n 


n=0 


N- 
2a eter ™ using (8.77) 
n=0 


N- 
-_ yf ge er = 
n=0 


it follows that the sequence {G,}"_ is periodic, with period N. We have therefore gen- 
erated a sequence of samples in the frequency domain that in some sense represents 
the spectrum of the underlying continuous-time signal. We shall postpone the question 
of the exact nature of this representation for the moment, but as the reader will have 
guessed, it is crucial to the purpose of this section. First, we consider the question of 
whether, from knowledge of the sequence {G,}*.) of (8.78), we can recover the 


original sequence { g,,}._) . To see how this can be achieved, consider a sum of the form 


N-1 ; 
S.=y¥G, ger. (N=1)srs0 (8.79) 


k=0 


Substituting for G, from (8.78), we have 


N-1 N-1 
aumtaor arser _ —jkA@(m+r)T 
S,= > ¥ ene Sn & 


k=0 \m=0 k=0 m=0 
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That is, on interchanging the order of integration, 


N-1 N-1 " : 
S, = »y Ln @ acne (8.80) 


m=0 k=0 
Now 


N-1 


—jkA@(m+r)T 
ae 


k= 


> 


is a geometric progression with first term e° = 1 and common ratio e!4°"*”"”, and so the 
sum to N terms is thus 
N-1 —jA@(mt+r)NT —j(mtr)20 
=jkA yr il l-e 
ye nom = = = 0 (me -r +n) 
—jA@(m4+r)T —jA@(m+r)T 
= l-e le 


When m =-r 


N-1 she ‘p —1 
= ae 
»y e? @(m+r) 1 N 


k= k=0 


Thus 
@ Aa (mtryT = Nb, (8.81) 


where 6,; is the Kronecker delta defined by 
l Ge 
oe G=J) 
0 (i#/) 
Substituting (8.81) into (8.80), we have 


N-1 


S.= NS £nO mr = Ng_, 


m=0 


Returning to (8.79) and substituting for S, we see that 
- vd 
which on taking n = —r gives 
g=t y eae (8.82) 
N k=0 


Thus (8.82) allows us to determine the members of the sequence 


Cae 


that is, it enables us to recover the time-domain samples from the frequency-domain 
samples exactly. 
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Solution 
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The relations 


N-1 ; 
Gay grr" (8.78) 
n=0 
all — jnkAoT 
B= >, Ge (8.82) 
k=0 


with Aw = 2n/NT, between the time- and frequency-domain sequences {g,}\_} and 


{G,}i9 define the discrete Fourier transform (DFT) pair. The pair provide pathways 
between time and frequency domains for discrete-time signals in exactly the same sense 
that (8.15) and (8.16) defined similar pathways for continuous-time signals. It should 
be stressed again that, whatever the properties of the sequences {g,,} and {G,} on the 
right-hand sides of (8.78) and (8.82), the sequences generated on the left-hand sides 
will be periodic, with period N. 


The sequence { g,}7_) = {1, 2, 1} is generated by sampling a time signal g(¢) at intervals 
with 7 = 1. Determine the discrete Fourier transform of the sequence, and verify that 
the sequence can be recovered exactly from its transform. 


From (8.78), the discrete Fourier transform sequence { G,};_) is generated by 


2 . 
Ge= Yi gerP" (k= 0,1,2) 


n=0 


In this case T= | and, with N = 3, (8.77) gives 


Thus 


2 : 2 
Gy = ¥ een = > g, =H=2Qt+2,+H= 14+24+1=4 


n=0 n=0 


2 
—jnx1x2n/3 -j -j -j =j 
G, = Yi Bre nx lx2t = ge + 2,773 4 ge iF = 1+2e j2m/3 le j4n/3 
n=0 


= e i2n3 (e?"3 4 2 & @ 20/3) = 2 e J2n3 el + cos ; Tt) = ei2n3 


2 2 
_ =jnx2x2n/3 _ =jn4n/3 0 —j4n/3 -~j8n/3 
G.= Signe = Vgc" =goe+ ge + ge 
n=0 


n=0 
= ein [ei™3 £2 4 eHn3] =2e 18 (1 + cos : qt) = ein 
Thus 
> r a 
{Gx hho = {4, € a ee 
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8.6.4 


We must now show that use of (8.82) will recover the original sequence {g,};_-). From 
(8.82), the inverse transform of {G,};_) is given by 


Bee 1 y Geiser 
yd 
again with T= 1, Aw= 7 and N = 3. Thus 
2 2 
Bo = FY Geer = PLP Ge = 3 (4 + PF + oH) 
k=0 k=0 
= + [4+ ee + 3") =! (4-2 cosin)=1 


2 


~ jkx1x21/3 j j 
gs, = oa G,e’ x1x20 = ! (G)+ G,e?"8 + Ge") 


3 


k=0 
=;(444+1+4+1)=2 
2 
B=lyV Gel? = 1 (G,+ Ge? + G, elt) 
k=0 


= 14+ emo 4 oi) = 1 (4-2c0s}n)=1 


That is 
ee = {1, 2, 1} a fp lea 


and thus the original sequence has been recovered exactly from its transform. 


We see from Example 8.19 that the operation of calculating N terms of the transformed 
sequence involved N x N= N? multiplications and N(NV — 1) summations, all of which 
are operations involving complex numbers in general. The computation of the discrete 
Fourier transform in this direct manner is thus said to be a computation of complexity 
N°. Such computations rapidly become impossible as N increases, owing to the time 
required for this execution. 


Estimation of the continuous Fourier transform 


We saw in Section 8.4.2 that the continuous Fourier transform provides a means of 
examining the frequency response of a stable linear time-invariant continuous-time 
system. Similarly, we saw in Section 8.6.2 how a discrete-time Fourier transform could 
be developed that allows examination of the frequency response of a stable linear time- 
invariant discrete-time system. By sampling this latter transform, we developed the 
discrete Fourier transform itself. Why did we do this? First we have found a way (at 
least in theory) of involving the computer in our efforts. Secondly, as we shall now 
show, we can use the discrete Fourier transform to estimate the continuous Fourier 
transform of a continuous-time signal. To see how this is done, let us first examine 
what happens when we sample a continuous-time signal. 

Suppose that f(f) is a non-periodic continuous-time signal, a portion of which is 
shown in Figure 8.24(a). Let us sample the signal at equal intervals T, to generate the 
sequence 
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Figure 8.24 

(a) Continuous- 
time signal f(t); 
(b) samples drawn 
from /(#). 
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fo 


O T NT t 
(b) 


{f0), f(T), --- f(T), «5 


as shown in Figure 8.24(b). Imagine now that each of these samples is presented in turn, 
at the appropriate instant, as the input to a continuous linear time-invariant system with 
impulse response A(t). The output would then be, from Section 5.6.6, 


co 


y(t) =| A(t — ooyacnar | h(t — t) f(t) 6(t— T) dt 


oo 


Pac | A(t — t)f(nT)6(t—nT)dtt+... 


=| ie oy, f(KT)5(t — kT) dt 


Thus - : 

y(t) = | : h(t — T)f,(7) dt (8.83) 
where - 

LD =, AUT) S(t- kT) = (NY. Ut ~ kT) (8.84) 


k=0 k=0 


which we identify as a ‘continuous-time’ representation of the sampled version of f(t). 
We are thus led to picture f(t) as in Figure 8.25. 
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Figure 8.25 
Visualization of f(t) 
defined in (8.84). 


In order to admit the possibility of signals that are non-zero for t < 0, we can 
generalize (8.84) slightly by allowing in general that 


F(t) = ft) ¥, 6(t - kT) (8.85) 


We can now use convolution to find the Fourier transform F(j@) of f(4). Using the 
representation (8.85) for f(t), we have 


R(j@) = Ff(O} = 710) ae Kn} 


k=—0o 


which, on using (8.61), leads to 


F(jo) = Fo) : =| 3 ste Kn} (8.86) 
where 

F{ f(O} = F(j@) 
From (8.54), 


7 ¥ au ar} = 22 § a( - 2a) 


k=-00 


so that, assuming the interchange of the order of integration and summation to be 
possible, (8.86) becomes 


F.(j0) = + (jo) «2% Y 5{@ - 28) 


k=-00 


a F(jlo - oY ao _ nh do’ 


= he F(jlo—- a/1)8( = 2th do’ 
= 72 Flilo- 7) 
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Thus 


— 2m 


- (8.87) 


FG) = 7, ¥ FUL@- kal), 


k=—0o 


Examining (8.87), we see that the spectrum F(jq@) of the sampled version f(t) of 
f(t) consists of repeats of the spectrum F'(jq@) of f(t) scaled by a factor 1/T, these repeats 
being spaced at intervals @) = 27/T apart. Figure 8.26(a) shows the amplitude spectrum 


Figure 8.26 

(a) Amplitude 
spectrum of a band- 
limited signal f(4); 
(b)—(e) amplitude 
spectrum | F,( ja) | 
of f£,(), showing 
periodic repetition 
of [F,(ja)| and (a) 
interaction effects 
as T increases. 


-2n/T mn 0 Om — 2n/T o 


—Wm 


-2wT OF 2m OM w 
(d) 


Om —2n/TO 2w/T Om ie 


(e) 
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Figure 8.27 
Sampling of a 
continuous-time signal. 


|F(ja@)| of a band-limited signal f(A); that is, a signal whose spectrum is zero for |@| > @,,. 
Figures 8.26(b—e) show the amplitude spectrum |F(j@)| of the sampled version for 
increasing values of the sampling interval 7. Clearly, as T increases, the spectrum of 
F(j@), as observed using |/(j@)| in —@,,< @ <@,,, becomes more and more misleading 
because of ‘interaction’ from neighbouring copies. 

As we saw in Section 8.6.2, the periodicity in the amplitude spectrum |,(jq@)| of £(4) 
is inevitable as a consequence of the sampling process, and ways have to be found to 
minimize the problems it causes. The interaction observed in Figure 8.26 between the 
periodic repeats is known as aliasing error, and it is clearly essential to minimize this 
effect. This can be achieved in an obvious way if the original unsampled signal /(f) is 
band-limited as in Figure 8.26(a). It is apparent that we must arrange that the periodic 
repeats of |F(j@)| be far enough apart to prevent interaction between the copies. This 
implies that we have 


Oy) = 204 

at an absolute (and impractical!) minimum. Since @, = 27/T, the constraint implies that 
TSt/On 

where 7 is the interval between samples. The minimum time interval allowed is 
Tin = T/O mm 


which is known as the Nyquist interval and we have in fact deduced a form of the 
Nyquist-Shannon sampling theorem. If 7 < T7,,;, then the ‘copies’ of F(jq@) are 
isolated from each other, and we can focus on just one copy, either for the purpose of 
signal reconstruction, or for the purposes of the estimation of F(j@) itself. Here we are 
concerned only with the latter problem. Basically, we have established a condition 
under which the spectrum of the samples of the band-limited signal f(A, that is the 
spectrum of £(4), can be used to estimate F(j@). 

Suppose we have drawn N samples from a continuous signal f(¢) at intervals 7, in 
accordance with the Nyquist criterion, as in Figure 8.27. We then consider 


Alt) =, fURT)B(t ~ kT) 


k=0 
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or equivalently, the sequence 
{files Where f,=(kT) 
Note that 
fM=0 E> (W-DT) 
so that 
fe=9 (k>N-1) 
The Fourier transform of f,(f) is 


co 


F,(j@) | Fine at =| % f(kT)6(t — kT) 03 dt 


—co k=0 


N-1 


= [pera enema 


k=0 
N-1 a N-1 _ 

=V f(kT)e" = »y Se (8.88) 
k=0 k=0 


The transform in (8.88) is a function of the continuous variable @, so, as in (8.78), we 
must now sample the continuous spectrum F,(jq@) to permit computer evaluation. 

We chose N samples to represent f(f) in the time domain, and for this reason we also 
choose NV samples in the frequency domain to represent F(j@). Thus we sample (8.88) 
at intervals Aw, to generate the sequence 


{F(jn Ao) ty (8.89a) 
where 
N-1 
F(jndo) = free" (8.89b) 
k=0 


We must now choose the frequency-domain sampling interval Aw. To see how to do 
this, recall that the sampled spectrum F,(j@) consisted of repeats of F(jq@), spaced at 
intervals 21/T apart. Thus to sample just one copy in its entirety, we should choose 


NAQO= 2n/T 
or 
Ao = 2n/NT (8.90) 


Note that the resulting sequence, defined outside 0 < n < N — 1, is periodic, as 
we should expect. However, note also that, following our discussion in Section 8.6, 
the process of recovering a time signal from samples of its spectrum will result in 
a periodic waveform, whatever the nature of the original time signal. We should not be 
surprised by this, since it is exactly in accordance with our introductory discussion in 
Section 8.1. 

In view of the scaling factor 1/T in (8.87), our estimate of the Fourier transform 
F(jo) of f(2) over the interval 


0<ts<(N-1)T 
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Example 8.20 


Figure 8.28 
The delayed 
triangular pulse. 


Solution 


will, from (8.89), be the sequence of samples 
{TR( jn do) } n= 
where 
N-1 - 
TR(jndo)= TY freer" 
k=0 
which, from the definition of the discrete Fourier transform in (8.78), gives 
TE(jnAq) = T x DFT {f,} 


where DFT { 7,} is the discrete Fourier transform of the sequence { f,}. We illustrate the 
use of this estimate in Example 8.20. 


The delayed triangular pulse f(f) is as illustrated in Figure 8.28. Estimate its Fourier 
transform using 10 samples and compare with the exact values. 


f(y) 


0.5 


Using N= 10 samples at intervals T= 0.2 s, we generate the sequence 


{feti-o = {£0), (0.2), (0.4), £(0-6), f0.8), (U.0), f1-2), fUL-4), f.6), £18} 
Clearly, from Figure 8.28, we can express the continuous function f(A) as 
t (0 <ft < 0.5) 
fO=41-¢ (5<—7=< 1) 
0 (t = 1) 
and so 
1 ite = 100.2, 0.4, 0.4, 0:2, 0,0; 00,01 


Using (8.78), the discrete Fourier transform {F,}?_, of the sequence { f,};- is 
generated by 


9 
_ -jkn MoT 20 2m 
i= ye , Where aaa eOr Wa 


k=0 
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That is, 


9 
7 —jkn (0.2m) 
ios = >. tik e 
k=0 


or, since ff =fs=fo=h =he=fo = 9, 
= jnk(0.27) 
F, = > feo” 270 
k=1 


The estimate of the Fourier transform, also based on N = 10 samples, is then the 
sequence 


{TF,}j-0 = {0.2F,}y-0 

We thus have 10 values representing the Fourier transform at 
@=nAq@ (n=0,1,2,...,9) 

or since A@ = 21/NT 
@=0, 1, 27, ..., 90 

At @=T, corresponding to n = 1, our estimate is 


4 : 
0.2F, = 0.25 fi ow 


k=1 
= 0.2[0.2 e107 + 0,4(e IO + eIO) + 0,2 e108] 
=-0.1992) 
At @= 2m, corresponding to n = 2, our estimate is 


4 . 
0.2F, = 0.25 fer 


k=1 
= 0.2[0.2 e210 + 0.4(e 108 + eI) + 0.2 -)] 
= —-0.1047 
Continuing in this manner, we compute the sequence 
{0.2F, 0.2F;,..., 0.2F,,} 
as 


{0.2400, —0.1992j, —0.1047, 0.0180}, —0.0153, 0, —0.0153, —0.0180j, 
~0.1047, 0.1992)} 


This then represents the estimate of the Fourier transform of the continuous function 
f(t). The exact value of the Fourier transform of f(f) is easily computed by direct use of 
the definition (8.15) as 


F(jo) = AL f(O} = 5 sine’ i @ 


which we can use to examine the validity of our result. The comparison is shown in 
Figure 8.29 and illustrated graphically in Figure 8.30. 
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Figure 8.29 
Comparison of 
exact results and 
DFT estimate for the 
amplitude spectrum 
of the signal of 
Example 8.20. 


Figure 8.30 

Exact result | F(j@) | 
(x) and DFT 
estimate TF, (L)) 
of the Fourier 
transform in 
Example 8.20. 


o Exact F(j@) DFT estimate |F(jo)| |DFT estimate| % error 


0 0.2500 0.2400 0.2500 0.2400 4% 
T —0.2026j -0.1992j 0.2026 0.1992 1.7% 
Qn —0.1013 ~0.1047 0.1013 0.1047 3.2% 
30 0.0225} 0.0180j 0.0225 0.0180 20% 
An 0 ~0.0153 0 0.0153 = 

Sn —0.0081j 0 0.0081 0 = 

6n -0.0113 —0.0153 0.0113 0.0153 = 

Tn 0.0041 —0.0180j 0.0041 0.0180 = 

8 0 ~0.1047 0 0.1047 = 

On —0.0025j 0.1992; 0.0025 0.1992 = 
IFGa)l. 


TF, 


0.2 


0.1 


O mn 2n 3n 4m Sn O6n Tn 8x On 


From the Nyquist-Shannon sampling theorem, with 7 = 0.2s, we deduce that our 
results will be completely accurate if the original signal /(¢) is band-limited with a zero 
spectrum for |@| > |@,,| = 5a. Our signal is not strictly band-limited in this way, and 
we thus expect to observe some error in our results, particularly near @ = 51, because 
of the effects of aliasing. The estimate obtained is satisfactory at m = 0, 7, 27, but 
begins to lose accuracy at @= 31. Results obtained above w= 57 are seen to be images 
of those obtained for values below @ = 5m, and this is to be expected owing to the 
periodicity of the DFT. In our calculation the DFT sequence will be periodic, with 
period NV = 10; thus, for example, 


| TF,| = | TF>10| = |TF.| = T|F3| 


As we have seen many times, for a real signal the amplitude spectrum is symmetric about 
@ = 0. Thus |F'3| = |], |.;| = |], and so on, and the effects of the symmetry are 
apparent in Figure 8.29. It is perhaps worth observing that if we had calculated (say) 
{TF.4, TF;,..., TF, TF, ..., TF;}, we should have obtained a ‘conventional’ plot, with 
the right-hand portion, beyond @ = 5z, translated to the left of the origin. However, 
using the plot of the amplitude spectrum in the chosen form does highlight the source 
of error due to aliasing. 


In this section we have discussed a method by which Fourier transforms can be 
estimated numerically, at least in theory. It is apparent, though, that the amount of labour 
involved is significant, and as we observed in Section 8.6.3 an algorithm based on this 
approach is in general prohibitive in view of the amount of computing time required. 
The next section gives a brief introduction to a method of overcoming this problem. 
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The fast Fourier transform 


The calculation of a discrete Fourier transform based on N sample values requires, as 
we have seen, N* complex multiplications and MN — 1) summations. For real signals, 
symmetry can be exploited, but for large N, } N* does not represent a significant improve- 
ment over N? for the purposes of computation. In fact, a totally new approach to the 
problem was required before the discrete Fourier transform could become a practical 
engineering tool. In 1965 Cooley and Tukey introduced the fast Fourier transform 
(FFT) in order to reduce the computational complexity (J. W. Cooley and J. W. Tukey, 
An algorithm for the machine computation of complex Fourier series, Mathematics of 
Computation 19 (1965) 297-301). We shall briefly introduce their approach in this 
section: for a full discussion see E. E. Brigham, The Fast Fourier Transform (Prentice 
Hall, Englewood Cliffs, NJ, 1974), whose treatment is similar to that adopted here. 

We shall restrict ourselves to the situation where N = 2” for some integer y, and, 
rather than examine the general case, we shall focus on a particular value of y. In 
proceeding in this way, the idea should be clear and the extension to other values of 
y appear credible. We can summarize the approach as being in three stages: 


(a) matrix formulation; 
(b) matrix factorization; and, finally, 
(c) rearranging. 


We first consider a matrix formulation of the DFT. From (8.78), the Fourier trans- 
form sequence {G,})) of the sequence {g,}.j is generated by 


N-1 ; 
G, = »y g, gee (k= 0, i vo N= 1) (8.91) 
n=0 
We shall consider the particular case when y= 2 (that is, N = 2” = 4), and define 
W = e@P20N = goin? 


so that (8.91) becomes 


N-1 3 
G= Yew" = Yew" (k=0, 1, 2,3) 
n=0 n=0 


Writing out the terms of the transformed sequence, we have 
G=2V°+2,W°+2W +2,W 
G=2W°+2W'+oW +2,W 
G,=2W°+2W?+oW'+2,W® 
G;=2W°+2W+2W%+2,W 


which may be expressed in the vector—matrix form 


Go ww w w'l\g, 
G, ww w wile, 
= 0 2 4 6 (8.92) 
G: ww WwW Wiig, 
G, Ww WwW We Wig, 
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or, more generally, as 
G,=W"sg, 
where the vectors G, and g, and the square matrix W” are defined as in (8.92). The 


next step relates to the special properties of the entries in the matrix W”. Note that 
w" = w"?’, where p is an integer, and so 


Wt=W°=1 

w® = Ww 

W=W 

Thus (8.92) becomes 

Gy 11 1 ile 

G, 1 W'eWw willie, 

G,| |1 wow Ww? 
2 82 

G,| [1 Ww Ww Wile, 


Equation (8.93) is the end of the first stage of the development. In fact, we have so far 
only made use of the properties of the Nth roots of unity. Stage two involves the 
factorization of a matrix, the details of which will be explained later. 


Note that 
1 wo O0]]1 0 WwW O ia&@ & 4 
1w 0 0/}]}0 1 0 WwW) |1 ww w (8.94) 
00 1w7'll1 0 WwW O iwww ; 
0 0 1wWwiotow 1ww w' 


where we have used W* = W' and W° = | (in the top row). The matrix on the right-hand 
side of (8.94) is the coefficient matrix of (8.93), but with rows 2 and 3 interchanged. 
Thus we can write (8.93) as 


Gy 1 Ww 0 O}J/1 0 W Ollg, 
G5 1 Ww 0 O}]0 1 0 Wi) ¢ 
a 1 2 : (8.95) 
G, 0 0 1 Wil OW Ole, 
G; 0 0 1 wWiol 0 Wile, 
We now define a vector g’ as 
| |1 0 W 0 |g, 
‘ 01 0 Ww 
2’ = - = ; 81 (8.96) 
&2 10 W 90 |) % 
gv} [0 1 0 W'lg, 


It then follows from (8.96) that 
8 =8. t+ We, 
gi =e, + W's; 
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so that g; and g are each calculated by one complex multiplication and one addition. Of 

course, in this special case, since W° = 1, the multiplication is unnecessary, but we are 

attempting to infer the general situation. For this reason, W° has not been replaced by 1. 
Also, it follows from (8.96) that 


2 =Z)+ We, 
8 =2,+ We, 


and, since W* = —W’°, the computation of the pair g and g, can make use of the com- 
putations of W°g, and W°g,, with one further addition in each case. Thus the vector g’ 
is determined by a total of four complex additions and two complex multiplications. 

To complete the calculation of the transform, we return to (8.95), and rewrite it in 
the form 


1 Ww 0 Olle 
G 1 wo 0 , 
Malan 0 1 ie om 
1 §2 
3 , 
G 0 0 1 Wiig 


It then follows from (8.97) that 
Gy = go + W's} 
Gy = go + W’gi 
and we see that G, is determined by one complex multiplication and one complex addition. 


Furthermore, because W? = —W’°, G, follows after one further complex addition. 
Similarly, it follows from (8.97) that 


G, = 92 + W'g; 
G;= 953+ Wg; 


and, since W* = —W', a total of one further complex multiplication and two further 
additions are required to produce the re-ordered transform vector 


[G G G Gr 


Thus the total number of operations required to generate the (re-ordered) transform is four 
complex multiplications and eight complex additions. Direct calculation would have 
required N? = 16 complex multiplications and N(N — 1) = 12 complex additions. Even 
with a small value of N, these savings are significant, and, interpreting computing time 
requirements as being proportional to the number of complex multiplications involved, 
it is easy to see why the FFT algorithm has become an essential tool for computational 
Fourier analysis. When N = 2’, the FFT algorithm is effectively a procedure for produc- 
ing yN x N matrices of the form (8.94). Extending our ideas, it is possible to see that 
generally the FFT algorithm, when N = 2”, will require i Ny (four, when N = 2? = 4) 
complex multiplications and Ny (eight, when NV = 4) complex additions. Since 


Y = log,N 


the demands of the FFT algorithm in terms of computing time, estimated on the basis 
of the number of complex multiplications, are often given as about N log, N, as opposed 
to N° for the direct evaluation of the transform. This completes the second stage of our 
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task, and we are only left with the problem of rearrangement of our transform vector 
into ‘natural’ order. 

The means by which this is achieved is most elegant. Instead of indexing Gp, G,, 
G;, G, in decimal form, an alternative binary notation is used, and[G, G, G, G,]" 
becomes 


[Goo Gor Gio Gul 
The process of “bit reversal’ means rewriting a binary number with its bits or digits in 
reverse order. Applying this process to [Go) Gy: Gio G,]' yields 

[Goo Gio Gor Gil’ =[G G, G, G;]' 
with decimal labelling. This latter form is exactly the one obtained at the end of the FFT 
calculation, and we see that the natural order can easily be recovered by rearranging the 
output on the basis of bit reversal of the binary indexed version. 

We have now completed our introduction to the fast Fourier transform. We shall now 


consider an example to illustrate the ideas discussed here. We shall then conclude by 
considering in greater detail the matrix factorization process used in the second stage. 


Example 8.21 Use the method of the FFT algorithm to compute the Fourier transform of the sequence 
{e.)5-0 = il, 2, 1, 0} 


Solution In this case N= 4 = 2’, and we begin by computing the vector g’ =[gj, 2, 295 23)’, 
which, from (8.96), is given by 


10 WwW 0flg 
ee 01 0 Wiig, 
"11 0 W Ollg, 
0 1 0 Wiig; 
For N=4 
w" = (eo?) = edn2 
and so 
1 0 1 O;) 1 2 
, 0 1 0 1/|2 2 
& no = 
1 0 -l 0\/1 0 
0 1 0 -1]/0 2 
Next, we compute the ‘bit-reversed’ order transform vector G’, say, which from (8.97) 
is given by 
1 Ww 0 0|lg% 
1 0 u 
G’ = Ww 0 &1 
0 O 1 W || 2 
0 0 1 Wil g% 
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or, in this particular case, 


Goo} | 1 1 0 O7;2 f 

G’= Go}}l -1 0 Of) 2) _ 0 (8.98) 
Goul}O 0 1 -j|]0 —2j 
GiljjO OO 1 j}i2 2) 


Finally, we recover the transform vector G=[G), G; G, G,]' as 


and we have thus established the Fourier transform of the sequence {1, 2, 1, 0} as the 
sequence 


{4, —2j, 0, 23} 


It is interesting to compare the labour involved in this calculation with that in 
Example 8.19. 


To conclude this section, we reconsider the matrix factorization operation, which is 
at the core of the process of calculating the fast Fourier transform. In a book of 
this nature it is not appropriate to reproduce a proof of the validity of the algorithm 
for any N of the form N= 2”. Rather, we shall illustrate how the factorization we intro- 
duced in (8.94) was obtained. The factored form of the matrix will not be generated 
in any calculation: what actually happens is that the various summations are performed 
using their structural properties. 

From (8.91), with W =e", we wish to calculate the sums 


G, = yg" k=0,1,...,N-1 (8.99) 


In the case N= 4, y= 2 we see that k and n take only the values 0, 1, 2 and 3, so we 
can represent both k and n using two-digit binary numbers; in general y-digit binary 
numbers will be required. 

We write k = k,k) and n = n,n, where ky, k,, ny) and n, may take the values 0 or | 
only. For example, k= 3 becomes & = 11 and n = 2 becomes n = 10. The decimal form 
can always be recovered easily as k= 2k, + ky and n = 2n, + no. 

Using binary notation, we can write (8.99) as 


1 1 
Git = » eS Sain WV 


ng=0 n,=0 


(2, +N9)(2k, +ko) 


(8.100) 


The single summation of (8.99) is now replaced, when y = 2, by two summations. Again 
we see that for the more general case with N = 2” a total of ysummations replaces the 
single sum of (8.99). 


www.z2Ofile.org 


698 THE FOURIER TRANSFORM 


The matrix factorization operation with which we are concerned is now achieved by 
considering the term 
(21, +Nng)(2k, +ko) 


in (8.100). Expanding gives 
(2n,+n9)(2k, +kg) _ py 2k tko)2m yk Ho)"9 


Any ky 2nyky (2k, +ko)Ng 


=W' Ww (8.101) 


Since W= e", and N= 4 in this case, the leading term in (8.101) becomes 
wen - go - aye 

Sul 

Again we observe that in the more general case such a factor will always emerge. 


Thus (8.101) can be written as 


(2, +N9)(2k, +ko) 2nyko (2k, +k) 
W 


=W 
so that (8.100) becomes 
= wo roo 
Gik, Pad . y Cia e (8.102) 
No=0 | n,=0 


which is the required matrix factorization. This can be seen by writing 


, 
8k ny. = Snjn 
ovo 1 ae 


n,=0 


wo 


(8.103) 


so that the sum in the square brackets in (8.102) defines the four relations 


S00 = ZW ° + gow = Zo + ZW" 

ae +20 =e + 21W" 8.104 
P. 2.0.1 54d 2 (8.104) 

Zio = Zo0W + Z1)W = Zo + S10W 

gu = gow" + gw" = Zo + guWw 


which, in matrix form, becomes 


Zoo 1 0 W 0 800 
: 1 w° 
a ie Bos (8.105) 
S10 1 0 W 0 10 
gi 0 1 0 Ww Sil 


and we see that we have re-established the system of equations (8.96), this time with 
binary indexing. Note that in (8.104) and (8.105) we distinguished between terms in W° 
depending on how the zero is generated. When the zero is generated through the value 
of the summation index (that is, when , = 0 and thus a zero will always be generated 
whatever the value of y) we replace W° by 1. When the index is zero because of the 
value of kj, we maintain W° as an aid to generalization. 
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The final stage of the factorization appears when we write the outer summation of 
(8.102) as 


1 
‘ r (2k, +k,))n 
Gi, = Da Beat — (8.106) 
ng=0 
which, on writing out in full, gives 
Goo = 20W° + gW™' = got gW° 
Gor = 20W> + g,W' = got BW? 
Gio = g10W" + oi, W"' = got oW' 
Gi, = gpW*?+ of W*' =210+ oW 


or, in matrix form, 


Gc 1 WwW’ 0 Off go 
; 1 Ww 0 0 : 
7 =|o 0 1 ow =u (8.107) 
10 S10 
Gi, 0 0 1 Wil gi, 


The matrix in (8.107) is exactly that of (8.97), and we have completed the factorization 
process as we intended. Finally, to obtain the transform in a natural order, we must 
carry out the bit-reversal operation. From (8.102) and (8.105), we achieve this by simply 
writing 


G42. (8.108) 


We can therefore summarize the Cooley—Tukey algorithm for the fast Fourier trans- 
form for the case N = 4 by the three relations (8.103), (8.106) and (8.108), that is 


1 
2njk, 
re 1%0 
Skyno Pe YS oe 
n,=0 
: (2k, +k))n 
eo , 1+) 
Gi, = Big 
0 


No= 


cs , 
Gai, = Gigk, 


The evaluation of these three relationships is equivalent to the matrix factorization 
process together with the bit-reversal procedure discussed above. 

The fast Fourier transform is essentially a computer-orientated algorithm and highly 
efficient codes are available in MATLAB and other software libraries, usually requiring 
a simple subroutine call for their implementation. The interested reader who would 
prefer to produce ‘home-made’ code may find listings in the textbook by Brigham 
quoted at the beginning of this section, as well as elsewhere. 
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8.6.6 Exercises 


Calculate directly the discrete Fourier transform of 
the sequence 


{1, 0, 1, 0} 


using the methods of Section 8.6.3 (see Example 8.19). 


Use the fast Fourier transform method to calculate 
the transform of the sequence of Exercise 28 
(follow Example 8.21). 


Use the FFT algorithm in MATLAB (or an 
alternative) to improve the experiment with 


k=4k,+2k, + ky, 
n=4n,+2n, +n, 


to show that 


k,=0 or 1 


n, = 0 or 1 


1 
Akyn 
x = 0n2 
Skynyny = y Snynyn,V 


n=0 


1 
(2k, +k,)2n 
” = , p then 
Sk kn. = y Signin V 


or1'o 
n,=0 


for alli 


for alli 


ao the estimation of the spectrum of the signal of 


Example 8.20. 


31 Derive an FFT algorithm for N = 2° = 8 points. 
Work from (8.99), writing 


1 
(4k,+2k, +k))n 

, _ ” wo? 1*q)"0 

Gyo kk, Sei 

ng=0 


=< , 
Gash = Fhgky hy 


wae airdlistlalarearlyeieliveim the design of analogue filters 


1H(jo)l 


“WU QO We w 


Figure 8.31 Amplitude 
response of an ideal 
low-pass filter. 


In this section we explore the ideas of mathematical design or synthesis. We shall 
express in mathematical form the desired performance of a system, and, utilizing the 
ideas we have developed, produce a system design. 

This chapter has been concerned with the frequency-domain representation of 
signals and systems, and the system we shall design will operate on input signals 
to produce output signals with specific frequency-domain properties. In Figure 8.31 we 
illustrate the amplitude response of an ideal low-pass filter. This filter passes perfectly 
signals, or components of signals, at frequencies less than the cut-off frequency @,. 
Above @,, attenuation is perfect, meaning that signals above this frequency are not 
passed by this filter. 

The amplitude response of this ideal device is given by 


1 < 
Go| -| (jal < @,) 
0 (ja| > @,) 


Such an ideal response cannot be attained by a real analogue device, and our design 
problem is to approximate this response to an acceptable degree using a system that 
can be constructed. A class of functions whose graphs resemble that of Figure 8.31 is 
the set 


1 


|2(10) | =-—————__-> 
\[1 + (@/@,)""] 


and we see from Figure 8.32, which corresponds to @, = 1, that, as n increases, the 
graph approaches the ideal response. This particular approximation is known as the 
Butterworth approximation, and is only one of a number of possibilities. 
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_; OY w 


To explore this approach further, we must ask the question whether such a response 
could be obtained as the frequency response of a realizable, stable linear system. We 
assume that it can, although if our investigation leads to the opposite conclusion then 
we shall have to abandon this approach and seek another. If H,(j@) is the frequency 
response of such a system then it will have been obtained by replacing s with jq@ in the 
system Laplace transfer function. This is at least possible since, by assumption, we are 
dealing with a stable system. Now 


tai 2 1 
| He(jo) |? = 7 


1+ (jo@/jo.) 


where |H,(j@)|’ = Hy(j@) H3(j@). If H,(s) is to have real coefficients, and thus be 
realizable, then we must have H}( j@) = H(—jo). Thus 
: Boom 1 _ 1 
H390)0,(- 10) = > = 


l +4 (@/@,)" 2n 


1+ (j@/jo,) 


and we see that the response could be obtained by setting s = jq@ in 


H,(s)H,(-s) = ——-— 


1 + (s/j@,) 


Our task is now to attempt to separate H,(s) from H,(—s) in such a way that H,(s) 
represents the transfer function of a stable system. To do this, we solve the equation 


1 + (s/jo, " =0 
to give the poles of H,(s)H,(—s) as 
s= @, ele Da2mn2] (k= 0, 1,2,3,...) (8.109) 


Figure 8.33 shows the pole locations for the cases n = 1, 2, 3 and 5. The important 
observations that we can make from this figure are that in each case there are 27 poles 
equally spaced around the circle of radius @, in the Argand diagram, and that there are 
no poles on the imaginary axis. If s=s, is a pole of H,(s)H,(—s) then so is s =—s,, and 
we can thus select as poles for the transfer function H,(s) those lying in the left half- 
plane. The remaining poles are then those of H,(—s). By this procedure, we have 
generated a stable transfer function H,(s) for our filter design. 

The transfer function that we have generated from the frequency-domain specification 
of system behaviour must now be related to a real system, and this is the next step 
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Figure 8.33 


Pole locations for the 


Butterworth filters: 


(O)n=1,()n=2; 


(X)n=3; 
(x)n=8. 


in the design process. The form of the transfer function for the filter of order n can be 
shown to be 


n 


o 
ee 
mw (sg = ss — 2)... (8 = 5,) 
where 5), 55, ... , 8, are the stable poles generated by (8.109). The reader is invited to 


show that the second-order Butterworth filter has transfer function 
2 
QO. 
H,(s) = ———— 
s+ \20.5+ @, 
Writing Y(s) = H,(s)U(s), with H,(s) as above, we obtain 
2 
Y(s) = 5 aaa OEE 
s+ \2@.5+ @; 


or 
(s? + /2@,8 + @2)¥(s) = 2 Us) (8.110) 


If we assume that all initial conditions are zero then (8.110) represents the Laplace 
transform of the differential equation 


2 
ara n 20282 + 02) = wuld) (8.111) 
t 


This step completes the mathematical aspect of the design exercise. It is possible to 
show that a system whose behaviour is modelled by this differential equation can be 
constructed using elementary circuit components, and the specification of such a circuit 
would complete the design. For a fuller treatment of the subject the interested reader 
could consult M. J. Chapman, D. P. Goodall and N. C. Steele, Signal Processing in 
Electronic Communications, Horwood Publishing, Chichester, 1997. 

To appreciate the operation of this filter, the use of the Signal Processing Toolbox in 
MATLAB is recommended. After setting the cut-off frequency @,, at 4 for example, the 
output of the system y(t) corresponding to an input signal w(t) = sint+ sin 10¢ will demon- 
strate the almost-perfect transmission of the low-frequency (@= 1) term, with nearly total 
attenuation of the high-frequency (@= 10) signal. As an extension to this exercise, the 
differential equation to represent the third- and fourth-order filters should be obtained, 
and the responses compared. Using a simulation package and an FFT coding, it is possible 
to investigate the operation of such devices from the viewpoint of the frequency domain 
by examining the spectrum of samples drawn from both input and output signals. 
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8.8.1 


Figure 8.34 
‘MATLAB’ M-file 
demonstrating 
frequency-domain 
filtering using the 

fast Fourier transform. 


frequency-domain filtering 


Introduction 


In this section we demonstrate the practical implementation of modulation, demodula- 
tion and frequency-domain filtering. These are the processes by which an information- 
catrying signal can be combined with others for transmission along a channel, with 
the signal subsequently being recovered so that the transmitted information can be 
extracted. When a number of signals have to be transmitted along a single channel at 
the same time, one solution is to use the method of amplitude modulation as described 
in Section 8.3.4. We assume that the channel is ‘noisy’, so that the received signal 
contains noise, and this signal is then cleaned and demodulated using frequency- 
domain filtering techniques. This idea is easy to describe and to implement, but cannot 
usually be performed on-line in view of the heavy computational requirements. Our 
filtering operations are carried out on the frequency-domain version of the signal, and 
this is generated using the fast Fourier transform algorithm. The MATLAB code in 
Figure 8.34 is designed to illustrate how results can be obtained working from basic 
ideas. The nature and usefulness of the Toolboxes now associated with MATLAB have 
made it possible to work at a higher level. Nevertheless it is thought valuable to retain 
this figure for instructional purposes, since it is easily modified. (Note: In Figure 8.34 1 
is used instead of j to denote /—1, to conform with MATLAB convention.) 


% Demonstration of frequency domain filtering using the FFT. 
% 

% 

% Some MATLAB housekeeping to prevent memory problems! 
clear 

clg 

% 

% Select a value of N for the number of samples to be taken. 

% Make a selection by adding or removing % symbols. 

% N must be a power of 2. 

%N = 512; 

N = 1024; 

%N = 2048; 

%N = 4096; 

%N = 8192; 

% 

% T is the sampling interval and the choice of N determines the 
% interval over which the signal is processed. Also, if 

% N frequency domain values are to be produced the resolution 
% is determined. 

T=0.001; 

t= 0:T:(N — 1)*T; 

delw = 2*pi/(T*N); 

% 

% Generate the ‘information’ 

f=t .*exp(-t/2); 

% 
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Figure 8.34 continued 


% Set the frequency of the carriers, we is the carrier which 
% will be modulated. 


we = 2«pix50; 
wea = 2*pi*120; 
% 


% Perform the modulation ... 
x = f. *cos(wext) + cos(weast); 


% 
% ...and add channel noise here 
nfac = 0.2; 


rand(‘normal’); 

x =x + nfac*rand(t); 

% 

% Plot the ‘received’ time signal 

plot(t,x) 

title(‘The time signal, modulated carrier and noise if added’) 
xlabel(‘time, t’) 

ylabel(‘x(t)’) 

pause 

% 

% Calculate the DFT using the FFT algorithm ... 

y = fft(x); 

z = Txabs(y); 

w = 0:delw:(N — 1)*delw; 

% 

% ... and plot the amplitude spectrum. 

plot(w,z) 

title(‘The amplitude spectrum. Spikes at frequencies of carriers’) 
xlabel(‘frequency, w’) 

ylabel(‘amplitude’) 

pause 

% 

% Construct a filter to isolate the information-bearing carrier. 
% 

% 2*hwind + | is the length of the filter ‘window’. 

% Set ffac to a value less than 1.0 ffac = 0.5 gives a filter 

% of half length we/2 where we is frequency of carrier. Don’t 
% exceed a value of 0.95! 

ffac = 0.5; 

hwind = round(ffac*wc/delw); 

] = 2*hwind + 1; 

% 

% Set the centre of the window at peak corresponding to we. 
% Check this is ok by setting | = 1! 

11 = round(we/delw) — hwind; 

% 

% Remember that we must have both ends of the filter! 
mask = [zeros(1,11),ones(1,l),zeros(1,N — (21 + 2*11 — 1)),ones(1,1),zeros(1,11 — 1)]; 
% 

% Do the frequency domain filtering... 

zz = mask.*y; 


% 

% ... and calculate the inverse DFT 
yya = ifft(zz); 

% 


% Remove rounding errors ... it is real! 


yy = 0.5*(yya + conj(yya)); 
% 
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Figure 8.34 continued ; . 
% Plot the ‘cleaned’ spectrum with only lower carrier present. 


plot(w,T*abs(zz)) 

title(‘Upper carrier eliminated and noise reduced’) 
xlabel(‘frequency, w’) 

ylabel(‘amplitude’) 

pause 

% 

% Now the signal is cleaned but needs demodulating so 
% form the product with 2*carrier signal ... 

dem = yy.*cos(we*t); 

dem = 2«dem; 

% 

% ... and take the DFT. 

demft = fft(dem); 

% 

% Use a low-pass filter on the result, the length is lp. 
% The same factor is used as before! 

llp = round(ffac*we/delw); 

masklp = [ones(1,llp),zeros(1,N — (2*Ilp — 1)),ones(1,llp — 1)]; 
% 

% Carry out the filtering... 

op = masklp.*demft; 


% 
% ... and plot the DFT of filtered signal. 
plot(w,T*abs(op)) 


title(“Result of demodulation and low-pass filtering’) 
xlabel(‘frequency, w’) 

ylabel(‘amplitude’ ) 

pause 

% 

% Return to the time domain... 

opta = ifft(op); 

opt = 0.5*(opta + conj(opta)); 

act—a: 

vp =N; 

% ... and finally plot the extracted signal vs the original. 
plot(t(1:vp),opt(1:vp),‘—’ ,t(.:vp),act(1:vp),*:’); 
title(‘The extracted signal, with original’) 
xlabel(‘time, t’) 

ylabel(‘f(t)’) 

pause 

% 

% Clean-up ... 

clg 

clear 

% 

% ... but responsibly! 

i= sqrt(—1); 

home 


8.8.2 Modulation and transmission 


We suppose that our ‘information’ consists of samples from the signal f(t) = te’, 


taken at intervals T= 0.001 s. This signal, or more correctly, data sequence, will be used 
to modulate the carrier signal cos (50*2*m*f). A second carrier signal is given by 
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Figure 8.35 
Time-domain 
version of 
noisy signal. 


x (2) 


8.8.3 


0.5 1.0 15 2.0 25 


Time, t 


cos (120*2*m*r), and this can be thought of as carrying the signal f(t) = 1. We 
combine these two signals and add ‘white noise’ to represent the action of the channel. 
This part of the exercise corresponds to the signal generation and transmission part of 
the overall process, and Figure 8.35 shows the time-domain version of the resulting 


signal. 


Identification and isolation of the information- 
carrying signal 


Here we begin the signal-processing operations. The key to this is Fourier analysis, and 
we make use of the fast Fourier transform algorithm to perform the necessary trans- 
forms and their inverses. First we examine the spectrum of the received signal, shown 
in Figure 8.36. We immediately see two spikes corresponding to the carrier signals, and 
we know that the lower one is carrying the signal we wish to extract. We must design 
a suitable filter to operate in the frequency domain for the isolation of the selected 
carrier wave before using the demodulation operation to extract the information. To do 
this, we simply mask the transformed signal, multiplying by 1 those components we 
wish to pass, and by 0 those we wish to reject. Obviously we want to pass the carrier- 
wave frequency component itself, but we must remember that the spectrum of the informa- 
tion signal is centred on this frequency, and so we must pass a band of frequencies 
around this centre frequency. Again a frequency-domain filter is constructed. We thus 
have to construct a bandpass filter of suitable bandwidth to achieve this, and moreover, 
we must remember to include the right-hand half of the filter! There are no problems 
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Figure 8.36 
Spectrum of 
received signal. 
Spikes and 
frequencies of 
carriers. 


8.8.4 


Amplitude 


1000 2000 3000 4000 5000 6000 7000 


Frequency, w 


here with the Nyquist frequency — at first glance we simply need to avoid picking up 
the second carrier wave. However, the larger the bandwidth we select, the more noise 
we Shall pass, and so a compromise has to be found between the necessary width for 
good signal recovery and noise elimination. Obviously, since we know the bandwidth 
of our information signal in this case, we could make our choice based on this know- 
ledge. This, however, would be cheating, because usually the exact nature of the trans- 
mitted information is not known in advance: if it were, there would be little point in 
sending it! In the M file we have set the half-length of the filter to be a fraction of the 
carrier frequency. The carrier frequency @, represents the maximum possible channel 
bandwidth, and in practice a channel would have a specified maximum bandwidth 
associated with it. Figure 8.37 shows the resulting spectrum after application of the 
bandpass filter, with a bandwidth less than @,. 


Demodulation stage 


The purpose of this operation is to extract the information from the carrier wave, and 
it can be shown that multiplying the time signal by cos @,7, where @, 1s the frequency 
of the carrier wave, has the effect of shifting the spectrum of the modulating signal so 
that it is again centred on the origin. To perform the multiplication operation, we have 
to return to the time domain, and this is achieved by using the inverse FFT algorithm. In 
the frequency-domain representation of the demodulated signal there are also copies of 
the spectrum of the modulating signal present, centred at higher frequencies (2@,, 4@,), 
and so we must perform a final low-pass filtering operation on the demodulated signal. 
To do this, we return to the frequency domain using the FFT algorithm again. The 
result of the demodulation and low-pass filtering operations is shown in Figure 8.38. 
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Figure 8.37 0.50 + ; >  — 
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8.8.5 Final signal recovery 


Frequency, 


The last operation to be performed is to return to the time domain to examine what we 
have achieved. After calling the inverse FFT routine, the extracted signal is plotted 
together with the original for comparison. The results with a fairly low value for the 
added noise are shown in Figure 8.39. If the process is carried out in the absence of 
noise altogether, excellent signal recovery is achieved, except for the characteristic 
‘ringing’ due to the sharp edges of the filters. 
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Figure 8.39 1.0 + : : : 


Extracted signal, 


shown together 
with the original 
signal. 


0 0.5 1.0 15 2.0 2.5 


Time, ¢ 


8.8.6 Further developments 


Readers are invited to develop this case study to increase their understanding. Try adding 
a second information signal modulating the second carrier wave, and extract both signals 
after ‘transmission’. Also add more carrier waves and modulating signals, and investigate 
signal recovery. If information signal bandwidths are limited to a fixed value, how 
many signals can be transmitted and recovered satisfactorily? What happens if 7 is 
altered? Can the ‘ringing’ effect be reduced by smoothing the transition from the string 
of ones to the string of zeros in the filter masks? Seek references to various window 
functions in signal-processing texts to assist in resolving this question. 


Cen aardlitsllaircarlyeieciiiem cirect design of digital filters 


and windows 


This application section provides a brief introduction to some methods of digital filter 
design. In particular we introduce a transform based on the Fourier transform itself, rather 
than going via the exponential form of Fourier series and the underlying periodicity 
implications. The material contained in this section first appeared in Signal Processing 
in Electronic Communication by M. J. Chapman, D. P. Goodall and N. C. Steele, originally 
published in the Horwood Series in Engineering Science in 1997 and is reproduced by 
courtesy of the current publishers Woodhead Publishing Limited. 


8.9.1 Digital filters 


Suppose f(f) is a signal with Fourier transform F(jq@) so that 
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f= | F(jaje'"da (8.112) 
20 


If we now sample f(t) at times t= kT, ke Z, we obtain the sequence {f,} = {f(KT)} and 
(8.112) gives 


ii x | F(ja)e!"" da (8.113) 


Splitting this infinite interval of integration into intervals of length 27/7, we obtain 


20 
a pty 
f= a > " F(jo)eda 
20 | on 
T 
2n i( 2m) ep 
me) Filer ee 
=— F\j| o+n= | Ie da 
an 2 | ale | 
Qn 
L |< (i( 74) jokT 
=— Fij}@tn=|\le da 
x | (ZMiloen 
j(w+n22 kT a 7 
since e = loTeinex — ei" As usual, we do not attempt to give conditions 


under which the above interchange between an integral and an infinite sum is valid. In 
any case, the above is only intended as a formal procedure leading to a definition for 
the discrete-time Fourier transform. 

If we now let 0 be the normalized frequency @= wT and set 


ie (O+n2n 
F(e’’) == rf Gea) 8.114 
(e") Tl NF ( ) 
where we note that the right-hand side has period 27 in 0, we obtain 
20 
a | F(e)e#d9 (8.115) 
20 ‘ 


The periodic function, F(e!°), is referred to as the discrete-time Fourier transform 
(DTFT) of the sequence {f,}. Equation (8.114) is unsuitable for calculation of the 
DTFT and so we instead use (8.115) to define the inverse DTFT and invert this in order 
to define the direct transform. We claim that the DTFT is, in fact, given by 


F(ei®) = y fen? = 3 fe?) (8.116) 


n=—0o n=—00 


Note that this is the same as the transform defined in (6.1), which is known as the 
bilateral z transform of {/} reflecting the fact that it is defined for both positive and 
negative values of the time index k, evaluated at z = e!’. This fact also explains the use 
of the notation, F(e!°). To show that (8.116) is valid, we substitute into the right-hand 
side of (8.115) to give 
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Example 8.22 


Solution 


2m 20 
1 j0) ,jkO 1 ‘ —jnd ,jké 
=-| FeeMdo=—-| YS feieldo 
al ae | Herre 


=+00 
Qo ” 


20 
= . ti ik) dQ 
yee | = 


assuming the interchange of summation and integration is permissible. However, it is 
easy to see that 


20 
al eil9 G9 = 0, fork#n =F 
20 1, fork=n . 


0 


and so the right-hand side of (8.115) reduces to 


co 


Y SP k-n =fis 


n=—0o 


as desired. To summarize, we have the two equations 


DIFT F(e*)= ¥ few (8.117a) 


k=—00 


20 
IDTFT hax | F(e®)ei!d@ (8.117b) 


0 


Calculate the discrete-time Fourier transform of the finite sequence 


{u} = {1, 2, 2, 1} 


We adopt the convention that the above sequence is ‘padded-out’ with zeros, that 
is we have {uw} = {u,} where uy = u; = 1, uw, = uy) = 2 and uw, = 0 otherwise. It follows 
from (8.117a) that 


U(ei?) = 1 + 2e? + 2677? + 3? 


-=j0 
=e : p cos(*P) +4 ca($) 
2 2 


A sketch of |U(e!®)| = ]2 cos(*2) +4 cos( 2) is given in Figure 8.40. |U(e”)| is called 


the amplitude spectrum of the sequence {uw}. Figure 8.40 clearly shows the periodicity 
of |U(e*)|, which by now is not surprising. In a similar fashion, we refer to arg U(e!”) 
as the phase spectrum of {1}. 
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Figure 8.40 
Amplitude 
spectrum |U(e!”)| 
for Example 8.22. 


|U(e”)| 


We are now in a position to develop a direct approach to the design of digital filters 
based on a Fourier series approach. Suppose that D(z) is the transfer function of a stable 
discrete-time system, then, we can write as usual, 


Y(z) = D(z)U(2) 


If the input sequence is {uw} = {6} = {1, 0, 0, 0, .. .}, the unit impulse sequence with 
z-transform U(z) = 1, then the transform of the output sequence, namely the impulse 
response sequence, is 


Y;(z) = D(z) = saz" 


Since the system is stable, by assumption, there is a frequency response which is 
obtained by taking the DTFT of the impulse response sequence. This is achieved by 
replacing z by ei in D(z) to obtain 


Dee!*") = De®) = > d,eir? (8.118) 
n=0 
where 0= aT. 
Now (8.118) can be interpreted as the Fourier expansion of D(e!®), using as basis 
functions the orthogonal set {e7”*?. It is then easy to show that the Fourier coefficients 
relative to this base are given by 


ak | D(e'® ei"? do 
m -1 


We now set D(e!”) to the desired ideal frequency response function and calculate the 
resulting Fourier coefficients, {7(n)} say. It should be noted that, at this stage, we can 
no longer restrict to n = 0, i.e. a(n), as defined above, is not causal and hence does not 
correspond with the impulse response of any realizable system. If a filter is to be real- 
ized using a finite number of delay elements, some form of truncation must take place. 
It is helpful to think of this truncation being performed by the application of a window, 
defined by a window weighting function w(n). The simplest window is the rectangular 
window, with weighting function w(n) defined by 


1 -n; <nsn 
wo =f 1 2 


0, otherwise 


Using this window, we actually form 


¥ winphdnje”® = > h,(nyew = Die!) 


n=-09 n=-ny 
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Example 8.23 


Solution 


where, if 7, and n) are sufficiently large, D(e!° ) will be an adequate approximation to 
D(e!®), the desired frequency response. It is important to note that the filter length, 
that is the number of delay elements or terms in the difference equation, depends on the 
choice of 1, and m2. This means that some accuracy will always have to be sacrificed in 
order to produce an acceptable design. 

We explore this technique by designing a low-pass filter in Example 8.23. 


Use the Fourier series, or direct design method, to produce a low-pass digital filter with 
cut-off frequency f: = | kHz, when the sampling frequency is f, = 5 kHz. 


We wish to make use of the non-dimensional frequency variable 6 and, since T= 1/f.= 
1/5000, we have 


6= of = 2nfT= 22L 
5000 


The cut-off frequency is then 0, = 27f./5000 = 27/5 and the ideal frequency response 
D(e®) is now defined by 


1 |O| < 2n/5 


De®) = 
0 6] > 2n/5 


We now calculate the coefficients h,(n) as 


h(n) = => | D(e®)e"*d0 
20 | 


TT 


= 2sine( 24) (also valid for n = 0) 


At this stage, we have to choose the length of the filter. By now, we know that a ‘long’ 
filter is likely to produce superior results in terms of frequency domain performance. 
However, experience again tells us that there will be penalties in some form or other. 
Let us choose a filter of length 9, with the coefficients selected for simplicity as sym- 
metric about n = 0. As already discussed, this choice leads to a non-causal system, but 
we deal with this problem when it arises. This scheme is equivalent to specifying the 
use of a rectangular window defined by 


és 1 -4<ns<4 
w(n) = 
0 otherwise 


We now calculate the coefficients ha(—4), ha(—3), . . . ha(0), . . . Aa(4), which are tabulated 
in Figure 8.41. 
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Figure 8.41 
Coefficients ha(h), 
for k=—4, -3,...,4. 


Figure 8.42 Amplitude 
response of the 
non-causal filter of 
Example 8.23. 


ha(+4) ha(+3) ha(#2) hg(#1) ha(0) 
—0.07568 —0.06237 0.09355 0.30273 0.40000 


The transfer function of the digital filter is then D, where 


Be = hse 


n=-4 
= —0.07568z~+ — 0.06237z% + 0.09355z~ + 0.302732! + 0.40000 
+ 0.30273z + 0.09355z? — 0.06237z? — 0.075682" 


Although this system is indeed non-causal, since its impulse response sequence con- 
tains terms in positive powers of z, we can calculate the frequency response as 


B(e!®) = -0.15137 cos(4) — 0.12473 cos(36) + 0.18710 cos(26) 
+ 0.60546 cos(@) + 0.40000 


Figure 8.42 illustrates the corresponding amplitude response. 


Figure 8.42, of Example 8.23, shows us that the amplitude response of our filter is a 
reasonable approximation to the design specification. We do, however, notice that there 
are some oscillations in both pass- and stop-bands. These are due to the abrupt cut-off 
of the rectangular window function and the effect is known as Gibbs’ phenomenon. 
Thus, the window function generates additional spectral components, which are referred 
to as spectral leakage. A way of improving the performance in this respect is discussed 
in Section 8.9.2. The immediate problem is the realization of this non-causal design. To 
see how we can circumvent the difficulty, we proceed as follows. 

The transfer function we have derived is of the general form 


D(z) = y hjk)z* 


=z" [h,(-N) th(-N+l)z't+...+h,0)z%+...+A(N)2] 
Suppose that we implement the system with transfer function 
D(z) =z" D(z) 


which is a causal system. First we notice that, on setting z = e!””, the amplitude response 
[D(e!”")| is given by 


De") = 17] le'””)| = te*”) 
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Figure 8.43 A 
realization of the 
final system of 
Example 8.23. 


8.9.2 


Figure 8.44 
Rectangular window 
sequence. 


that is, it is identical with that of the desired design. Furthermore, 
arg{D(e!”")} = arg{D(e!”")} — Nor 


indicating a pure delay of amount NT in the response of the second system. This means 
that, assuming we are prepared to accept this delay, our design objective can be met by 
the system with transfer function D(z) given by 


D(z) = [-0.07568 — 0.062372 + 0.093552? + 0.30273z° + 0.40000z4 
+ 0.30273z° + 0.09355z° — 0.062372’ — 0.07568z *] 


It is evident from Figure 8.43 that the filter designed in Example 8.23 differs from the 
previous designs. The nature of this difference is the absence of feedback paths in the 
block diagram realization of Figure 8.43. One effect of this is that the impulse response 
sequence is finite, a fact which we already know, since the design method involved 
truncating the impulse response sequence. Filters of this type are known as finite 
impulse response (FIR) designs and may always be implemented using structures not 
involving feedback loops. Another name used for such structures is non-recursive, but 
it is not correct to assume that the only possible realization of an FIR filter is by use of 
a non-recursive structure; for details see M. T. Jong, Methods of Discrete Signals and 
Systems Analysis, McGraw-Hill, New York, 1982. 


{u(kT )} 
>| D > D >| D >| D >| D >| D D D 
| y | 
a f a 
ic v c 
x >> z 
> {y(kT)} 


a =—0.07568, b = —0.06237, c = 0.09355, d= 0.30273, f= 0.40000. 


Windows 


In this section, we consider the problem identified in Example 8.23 in connection with 
the sharp cut-off of the rectangular window function. 
The rectangular window sequence, illustrated in Figure 8.44, is defined by 


(| < N 
w(eyayt SA 
0 otherwise 


Ack) 


1 
@eeeeoenenenogeeneee 8 @ 
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Figure 8.45 DTFT of 
the rectangular window 
sequence. 


which can be expressed in the form 
w(k) = Uk + N) — (k-(N + 1)) 


where ¢(k) = {h(k)}, defined in Example (6.22). 
Since 


N+} -(N+4) 


the DTFT of the sequence {w(h)} is 
sin(;(2N + 1)@) 


Wel®)= = for 040 
sin(; @) 
_QN+ 1)sinc($(2N + 1)@) 
7 sinc(+6) 
; N 
It is easy to see that W(e!°)= W1) = p> w(n) = 2N + | and so the above formula, using the 
n=—N 


sinc function, is valid for all 0, including @= 0. The graph of this function is illustrated 
in Figure 8.45. The first positive (negative) zero in its spectrum is the positive (negative) 
value of @ closest to zero such that W(e!®) = 0. The main lobe of the window function 
is that part of the graph of W(e!®) that lies between the first positive and first negative 
zero in W(e!®). The main lobe width is the distance between the first positive and 
negative zeros in W(e!”). As the length of the window increases, the main lobe narrows 
and its peak value rises and, in some sense, W(e!®) approaches an impulse, which is 
desirable. However, the main disadvantage is that the amplitudes of the side lobes also 
increase. 


A W (e) 


The use of any window leads to distortion of the spectrum of the original signal caused 
by the size of the side lobes in the window spectrum and the width of the window’s 
main spectral lobe, producing oscillations in the filter response. The window function 
can be selected so that the amplitudes of the sides lobes are relatively small, with the 
result that the size of the oscillations is reduced; however, in general, the main lobe 
width does not decrease. Thus, in choosing a window, it is important to know the trade- 
off between having narrow main lobe and low side lobes in the window spectrum. 

A considerable amount of research has been carried out, aimed at determining suitable 
alternative window functions which smooth out straight truncation and thus reduce 
the Gibbs’ phenomena effects observed in the amplitude response of Figure 8.42. 
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Figure 8.46 Some 
popular window 
functions. 


Example 8.24 


Solution 


Figure 8.47 Hamming 
window coefficients 
for-4<k <4. 


To minimize the effect of spectral leakage, windows which approach zero smoothly at 
either end of the sampled signal are used. We do not discuss the derivation of the various 
window functions, rather we tabulate, in Figure 8.46, some of the more popular exam- 
ples in a form suitable for symmetric filters of length 2N + 1. For a more detailed 
discussion on windows and their properties, see, for example: E. C. Ifeachor and B. W. 
Jervis, Digital Signal Processing: A Practical Approach, Addison-Wesley, Wokingham, 
UK, 1993; A. V. Oppenheim and R. W. Schafer, Discrete-time Signal Processing, Prentice- 
Hall, Englewood Cliffs, NJ, 1989; S. J. Stearns and D. R. Hush, Digital Signal Analysis, 
Prentice-Hall, Englewood Cliffs, NJ, 1990. 


Window name w(k) 

Bartlett (kK+N)/N —N=S=k<0 
WO) ae O<k<N 

von Hann or Hanning w(k) = 0.5 + 0.5 cos(ak/(N + 1)) —NSk<N 

Hamming w(k) = 0.54 + 0.46 cos(k/N ) —NSk=<N 

Blackman w(k) = 0.42 + 0.5 cos(tk/N ) + 0.08 cos(27tk/N ) NSk=sN 


In each case, w(k) = 0 for k outside the range [—N, N]. 


Note: Slight variations on the above definitions may be found in various texts. These 
tend to involve switching between ‘division by N’, ‘division by N + 3’ and ‘division 
by N + 1’. For example, the von Hann or Hanning window is variously defined by 
w(k) = 0.5(1 + cos(1tk/N)) or w(k) = 0.5(1 + cos(20k/(2N + 1))) or w(A) = 0.5(1 + cos(stk/ 
(N + 1))) for |A| = N with w(k) = 0 for |k| > N. The Bartlett window, or one of its 
variations, is sometimes referred to as a triangular window. It should also be noted 
that both the Bartlett window and the Blackman window, as defined in Figure 8.46, 
satisfy w(—N) = w(N) = 0 and hence give rise to difference equations of order 2N — 2 
rather than 2N. 

Formulations for other configurations can easily be deduced, or may be found in, 
for example, L. B. Jackson, Digital Filters and Signal Processing, Kluwer Academic 
Publishers, Boston, MA, 1986; R. E. Ziemer, W. H. Tranter and D. R. Fannin, Signals 
and Systems, Macmillan, New York, 1983. The section closes with an example of the 
application to the design of Example 8.23. 


Plot the amplitude response for the filter design of Example 8.23, using (a) the 
Hamming window and (b) the Blackman window. 


(a) The transfer function coefficients are now given by h,(k) w,(k), where w,(k) are 
the Hamming window coefficients, calculated with N = 4 and —-4 < k < 4. The 
Hamming window coefficients are tabulated in Figure 8.47. 


= 
II 
& 


+4 +3 +2 +] 0 
0.08000 0.21473 0.54000 0.86527 1.00000 
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Figure 8.48 
Amplitude response of 
the filter of Example 
8.23, with Hamming 
window. 


Figure 8.49 
Blackman window 
coefficients for 
4<ks<4, 


Figure 8.50 
Amplitude response 
of the filter of 
Example 8.23, with 
Blackman window. 


(b) 


t t a 
2n/5 1 2n 0 


The transfer function then becomes 


D,,(z) = [-0.00605 — 0.013392 + 0.05052z7? + 0.26194z* + 0.400002 
+ 0.261942 + 0.05052z°— 0.013392’ — 0.00605z*] 


The frequency response is then obtained by writing z = e!°, as 


D,e'®) = e* (0.01211 cos(40) — 0.02678 cos(36) + 0.10103 cos(26) 
+ 0.52389 cos(6) + 0.40000) 


Figure 8.48 illustrates the magnitude of this response and the reduction of 
oscillations in both the pass- and stop-band is striking. The penalty is the lack 
of sharpness near the cut-off frequency, although the stop-band characteristics 
close to 9= 7 are quite good. 


Proceeding as in case (a), we calculate the Blackman window coefficients as 
shown in Figure 8.49. The Blackman windowed transfer function is thus 


D,(z) = 0.00414 + 0.031812! + 0.234182? + 0.40000z° + 0.234182+ 
+ 0.03181z%— 0.00414z% 


+4 +3 +2 +1 0 
0.00000 0.06645 0.34000 0.77355 1.00000 


and the frequency response is found as 


D(e!®) = e** (—0.00829 cos(3 0) + 0.06361 cos(20) 
+ 0.46836 cos(@) + 0.40000) 


The amplitude response is shown in Figure 8.50 and this design again suffers 
from a relatively poor performance in terms of sharpness of cut-off. The ripples 
observed in the pass- and stop-bands with the rectangular window have been 
removed as before. However, the ‘flat’ characteristic of the Hamming design close 
to @=17 is not evident when using the Blackman window for this particular filter. 
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8.9.3 Exercises 


Use the direct design method with a rectangular 
window of length 11 to produce a causal 
low-pass filter with non-dimensional cut-off 
frequency 


SS 
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g.=8 
~ 2 


Plot the frequency response. 


Repeat Exercise 32 but use a Hamming window. 


8.10 Review exercises (1-25) 


il 


2 


Calculate the Fourier sine transform of the causal 
function f(t) defined by 
t (0<¢<1) 
f(O)=41 (1<tS2) 
OQ @ Sy 


Show that if F{f()} = F(j@) then F{ f(-1)} = 
F(-j@). Show also that 


F{ f(t — a)} = e° F(-jo) 


where a is real and positive. 
Find ¥{ f(t)} when 


-in (t<-2) 
f(t)=\jmt (-2<t<2) 
i (PS 2) 


Use the result 
FM + 57) — H(t— 57) = T sinc or 


and the frequency convolution result to verify that 
the Fourier transform of the windowed cosine 
function 


f(t) = cos Mot [H(t + $7) — H(t- 3T)] 
is 

IT [sinc } (@— @,)T + sinc} (@+ @))T] 
Show that 

O(t — t;) * O(t — b) = d(t - (t, + )) 
and hence show that 


F{cos Mt H(t)} = 4 n[5(@+ @) + 6(@- @)] 
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Establish the demodulation property, 

Fi f(t)cos Wot cos Wot} 

= § F(j@) + § [FG@+t 2ja) + Fja@+ 2ja,)] 
Use the result #{H(t+ T) — H(t- T)} =2T sine oT 
and the symmetry property to show that 

F{sine t} = n[H(@+ 1) - H(@—- 1)] 


Check your result by use of the inversion integral. 


For a wide class of frequently occurring Laplace 
transforms it is possible to deduce an inversion 
integral based on the Fourier inversion integral. 
If X(s) = L{x(2)} is such a transform, we have 


Waslles 
Oe | X(s)e" ds 
4270 : 

Uae? 
where Re (s) = ¥, with y real, defines a line in the s 
plane to the right of all the poles of X(s). Usually 
the integral can be evaluated using the residue 
theorem, and we then have 


x(t) = > residues of X(s)e” at all 
poles of X(s) 


(a) Write down the poles for the transform 


1 
(s- a)(s- b) 


where a and b are real. Calculate the residues of 
X(s)e” at these poles and invert the transform. 


(b) Calculate 


(i) | : | Gi) £"\; L 
(s = 2) (ge ll) 


(c) Show that 


ear = tsint 


X(s) = 


720 
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THE FOURIER TRANSFORM 


A linear system has impulse response A(t), so that 
the output corresponding to an input u(t) is 


y(t) = | h(t- tT) u(t) dt 


When u(t) = cos Wot, y(f) = —SiIN Wot (Wy = 0). 
Find the output when u(t) is given by 


(a) cos @(t + } 1) 
(c) elo! (d) e 


This system is known as a Hilbert transformer. 


(b) sin @of 


-j@ot 


In Section 8.5.1 we established that 


where sgn(f) is the signum function. Deduce that 


13 
Fsgn(t)} = = 


and use the symmetry result to demonstrate that 


#-1} = jsgn(@) 
14 

The Hilbert transform of a signal f(t) is 

defined by 


1 Tt 
Fru(et) = HLA} = 7 | LO ar 
Tl E28 
Show that the operation of taking the Hilbert 
transform is equivalent to the convolution 


-4+4 40 15 
Tt 


and hence deduce that the Hilbert-transformed 
signal has an amplitude spectrum F};(j@) 
identical with f(f). Show also that the phase of 
the transformed signal is changed by +; 7, 
depending on the sign of a. 


Show that 


i 
(( +a°)(t- x) 


1 a a xt 
a 
xt+a\f+a t-x fta 


Hence show that the Hilbert transform of 


ft)= 5 (a> 0) 
ag 


hid 


is 


a 


2) 2. 
Ket 


If Fy(x) = A{f(O)} is the Hilbert transform 
of f(t), establish the following properties: 
(a) H{ flat t)} = Fux + a) 

(b) H{f(at)} =Fy(ax) (a > 0) 

(c) H{f(-at)} =—Fy(-ax) (a > 0) 


(@) x(a} = + Fiyi(x) 


(ec) stzf(t)} = Fila) +2 | f(t) dt 


Use Exercises (9) and (10) to deduce the inversion 
formula 


Define the analytic signal associated with the real 
signal f(t) as 


FAD = LO) — JF 


where F(t) is the Hilbert transform of f(t). Use 
the method of Review exercise 13 to show that 


Pea (@> 0) 


FAL f,(t)} = F,(j@) = 
(@<0) 
Use the result #{H(t)} = 1/j@ + 16(@) and the 
symmetry property to show that 


F' {H(@)} = 46(t) +L 
2nt 


(Hint: H(—@) = 1 — H(@).) 

Hence show that if f(t) is defined by ¥{f(t)} = 
2H(@)F( ja) then f(t) = f(t) — jF y(t), the analytic 
signal associated with f(t), where F(j@) = F{ f(t)} 
and Fyy(t) = H{ f(t)}. 

If f(t) = cos Wot (My > 0), find F{ f(A} and 
hence f(t). Deduce that 


HE{COS Wot} = —SiN Wot 


By considering the signal g(t) = sin Mot (Mo > 0), 
show that 


H{ Sin Wot} = COS Wot 


www.2Ofile.org 


16 


7 


18 


A causal system has impulse response /(f), 
where h(t) = 0 (t < 0). Define the even part 
h(t) of h(t) as 


h(t) = 5 (h(t) + h(-t)] 
and the odd part /,(¢) as 
Ast) = 5 [h) — h(-)] 
Since h(t) = 0 (t < 0) deduce that 
h(t) = sgn()h(2) 
and that 
A(t) =h(t) + sgn(Oht)_ for all t 


Verify this result for A() = sint H(t). Take the 
Fourier transform of this result to establish that 


A(j) = Hj) + jA{Hjo)} 


Let A(t) = &“H(t) be such a causal impulse 
response. By taking the Fourier transform, deduce 
the Hilbert transform pair 


a 

H} — =-— 
Ds 2 2 2 
att Oe EX: 


Use the result 


Ht AO} = xHL fl} + 1| f(t) dt 


to show that 
x - i} _ 2 
a+? x eg 


The Hartley transform is defined as 


F,(s) = H{f()} = | f(t) cas 2nst dt 


where cas ¢ = cos¢ + sint. Find the Hartley 
transform of the functions 


(a) fO=e*HD (a> 0) 


(le 90) 


(b) fi = 
‘ ( ss /00) 


An alternative form of the Fourier transform pair is 
given by 


F(ip) = | faye dt 
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g(t) = | G(jp)er"'de 
where the frequency p is now measured in hertz. 
Define the even part of the Hartley transform as 
E(s) = 5 [Fu(s) + Fu(-s)] 
and the odd part as 
O(s) = 5 LFuls) — Fu(-s)] 
Show that the Fourier transform of f(f) is given by 
F(jp) = E(p) — jO(p) 
and confirm your result for f(t) = e~H(#). 
Prove the time-shift result for the Hartley 
transform in the form 
H{f(t — T)} = sin(2nTs) Fy(-s) 
+ cos(2nTs) Fi,(s) 


Using the alternative form of the Fourier transform 
given in Review exercise 18, it can be shown that 
the Fourier transform of the Heaviside step 
function is 


FH (t)} = + +18(p) 
jJpu 
Show that the Hartley transform of H(f) is then 
50(s) + as 
ST 
and deduce that the Hartley transform of 
H(t — 4) is 


16(s) ae cos Ts - SIN TS 
7 ST 


Show that H{6(7)} = 1 and deduce that 
H{1} = 6(s). Show also that H{d(t— %)} = 
cas 27st, and that 


H{cas 21tsot} =H {cos 2st} + H{sin 2rsot} 
= 0(S — Sp) 

Prove the Hartley transform modulation theorem 
in the form 

H{ f(0) cos 2st} = ; Fi(s — 89) + 5 Fi(s = 5) 
Hence show that 

H{cos 27st} = § [5(s — so) + 5(s + 59)] 

H{sin 2sot} = 3 [6(s — 59) — 5(s + 5y)] 
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2S 


24 


25 


Show that 


- lol 
Te 


jo 


[in Consider | (1+ Py 7 


Ft tan 't} = 


Show that 
x(t) = } (1 + cos @pf)[H(t+ $7) — H(t- 37 )] 
has Fourier transform 


T[sinc @+ § sinc(@— @) + § sinc(@+ @)] 


The discrete Hartley transform of the sequence 
{ f(r) } 49 is defined by 


H(v) = iy ih) cas (a | 


(= 0 lh) 


The inverse transform is 


N-1 


fr) =) H(v) cas (222) Eola 


Show that in the case N= 4, 
H=Tf 
H=[H(0) H(1) H(2) H@)J" 
f=1f0) fA) f2) fT 


is wile yal al 
pols de Slo ai 
aii si 2 sil 

l sf sil J 


Hence compute the discrete Hartley transform of 
the sequence {1, 2, 3, 4}. Show that T* = } land 
hence that T~! = 4T, and verify that applying the 
T™ operator regains the original sequence. 
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724 PARTIAL DIFFERENTIAL EQUATIONS 


9.1 


Introduction 


In Chapter 5 we considered the role of ordinary differential equations in engineering. 
However, many physical processes fundamental to science and engineering are governed 
by partial differential equations, that is equations involving partial derivatives. The 
most familiar of these processes are heat conduction and wave propagation. To describe 
such phenomena, we make assumptions about gradients (for instance, the Fourier law 
that heat flow is proportional to temperature gradient) and we write down balance 
equations; partial differential equations are thus produced in a natural way. Unless the 
situation is very simple, there will be many independent variables, for example a time 
variable ¢ and a space variable x, and the differential equations must involve partial 
derivatives. 

The application of partial differential equations is much wider than the simple 
situations already mentioned. Maxwell’s equations (see Example 3.16) comprise a set 
of partial differential equations that form the basis of electromagnetic theory, and are 
fundamental to electrical engineers and physicists. The equations of fluid flow are partial 
differential equations, and are widely used in aeronautical engineering, acoustics, the 
study of groundwater flows in civil engineering, the development of most fluid handling 
devices used in mechanical engineering and in investigating flame and combustion 
processes in chemical engineering. Quantum mechanics is yet another theory governed 
by a partial differential equation, the Schrédinger equation, which forms the basis of 
much of physics, chemistry and electronic engineering. Stress analysis is important in 
large areas of civil and mechanical engineering, and again requires a complicated set of 
partial differential equations. This is by no means an exhaustive list, but it does illus- 
trate the importance of partial differential equations and their solution. 

One of the major difficulties with partial differential equations is that it is extremely 
difficult to illustrate their solutions geometrically, in contrast to single-variable problems, 
where a simple curve can be used. For instance, the temperature in a room, particularly 
if it is time-varying, is not at all easy to draw or visualize, but such information is of 
crucial importance to a heating engineer. Modern graphics packages have improved the 
situation considerably in two and three dimensions and the displays can often give a 
good qualitative understanding. A second basic problem with partial differential equations 
is that it is intrinsically more difficult to solve them or even to decide whether a solution 
exists. The driving force of most physical systems that can be modelled by partial dif- 
ferential equations is determined by either what happens on the boundary of the region 
under consideration or how the system is started at zero time. Boundaries, therefore, play 
a very significant role, and we shall see that a problem can have a solution for one set 
of boundary conditions but not for another. Finding a solution to a partial differential 
equation is often quite straightforward but finding the solution that fits the boundary 
conditions is very difficult. 

The solution of partial differential equations has been greatly eased by the use of com- 
puters, which have allowed the rapid numerical solution of problems that would otherwise 
have been intractable. Such methods have generally been integrated into this chapter, 
since they are now one of the standard techniques available. However, the finite-element 
method is considered separately, since it is more complicated, and requires a lot of 
careful thought and work (the section dealing with it can be omitted on a first reading). 
The finite-element method originated in stress analysis in civil engineering work, but 
has now spread into most areas where complicated boundaries are encountered. 
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There are three basic types of equation that appear in most areas of science and 
engineering, and it is essential to understand their solutions before any progress can be 
made on more complicated sets of equations, nonlinear equations or equations with 
variable coefficients. 


wa ~=General discussion 


The three basic types of equation are referred to as the wave equation, the heat- 
conduction or diffusion equation and the Laplace equation. In this section we briefly 
discuss the formulation of these three basic forms, and then consider each in more detail 
in later sections. The various sections will concentrate on finding and understanding 
solutions of the three types of equations in simple regions. The treatment of advanced 
methods, more complicated equations and other regions will be left to more compre- 
hensive books on partial differential equations (see, for example, R. Haberman, Applied 
Partial Differential Equations, Prentice Hall, Upper Saddle River, NJ, 2003). 


9.2.1 Wave equation 
e+e aVu (9.1) 
a Yi, 


Many phenomena that involve propagation of a signal require the wave equation (9.1) 
to be solved in the appropriate number of space dimensions. Perhaps the simplest, in 
one space dimension, is the vibration of a taut string stretched to a uniform tension T 
between two fixed points as illustrated in Figure 9.1(a), where w is the displacement, 
x is measured along the equilibrium position of the string and f is time. Applying 
Newton’s law of motion to an element As of the string (Figure 9.1b), for motion in the 
u direction, we have 


net force in wu direction = mass element x acceleration in wu direction 


that is, 


ou 
2 


Tsin(y + Ay) — Tsin y= pAs (9.2) 


Displaced position of string 


Equilibrium position of the string 


(a) (b) 


Figure 9.1 Displacement of an element of a taut string. 
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where p is the mass per unit length of the string. Neglecting terms quadratic in small 
quantities and using the Taylor series expansions 


cosAy = 1+ O(Ay’) ~ 1, sinAy=Ay+ O(Ay’) ~ Aw 


and the expression 


for the arclength, (9.2) becomes 


2, 
Tsin y+ Tcos wAy-— Tsin v= pares 
t 
or 
T cos yay = paAxZu 
Ax ot 
which in the limit as Ax > 0 becomes 
0 a 
T cos oe =p =e (9.3) 


Again assuming that y itself is small for small oscillations of the string, we have 
cos y = I, and the gradient of the string 


Ot = tan = 
ox and 


and hence from (9.3) we obtain 
pou 


2 
Ou 
~ 2 


ax ot 


Thus the displacement of the string satisfies the one-dimensional wave equation 


lca us (9.4) 

Cay ay 
and the propagation of the disturbance in the string is given by a solution of this 
equation where c? = T/p. 


By considering the theory of small displacements of a compressible fluid, sound 
waves can likewise be shown to propagate according to (9.1). The one-dimensional 
form (9.4) will model the propagation of sound in an organ pipe, while the spherically 
symmetric version of (9.1) will give a solution for waves emanating from an explosion. 
Because it is known that most wave phenomena satisfy the wave equation, it is reason- 
able, from a physical standpoint, that the propagation of electromagnetic waves will 
also satisfy (9.1). A careful analysis of Maxwell’s equations in free space is required 
to show this result (see Example 3.16). We could give further examples of physical 
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Example 9.1 


Solution 


Example 9.2 
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phenomena that have (9.1) as a basic equation, but we have described enough here to 
establish its importance and the need to look at methods of solution. An aspect of the 
wave equation that is not often discussed is its bad behaviour. Any discontinuities in a 
variable or its derivatives will, according to the wave equation, propagate with time. An 
obvious physical manifestation of this is a shock wave. When an aircraft breaks the 
sound barrier, a shock is produced and the sonic boom can be heard many miles away. 
How the shock is produced is a complicated nonlinear effect, but once it has been 
produced it propagates according to the wave equation. 


Show that 


Uu=u sin( ®) cos Bet) 
anes, Ce 2 £ 


satisfies the one-dimensional wave equation and the conditions 
(a) a given initial displacement u(x, 0) = uy sin(wx/L), and 


(b) zero initial velocity, Ou(x, 0)/ot = 0. 


Clearly the condition (a) is satisfied by inspection. If we now partially differentiate u 
with respect to ¢, 


Oe =. Mime sin (=) sin (<1) 
ot L L L 


so that at t= 0 we have Ou/ot = 0 and (b) is satisfied. 
It remains to show that (9.4) is also satisfied. Using the standard subscript notation 
for partial derivatives, 


Ox’ 2 
2 2 
Ou Uy C. (1X Tct 
iy = = = in| = | cos| — 
ot EE L EL 


so that the equation is indeed satisfied. 
This solution corresponds physically to the fundamental mode of vibration of a taut 
string plucked at its centre. 


Verify that the function 


u = aexp , (2 = ) 
7 hh 
satisfies the wave equation (9.4). Sketch the graphs of the solution wu against x at t= 0, 


t=2h/c and t= 4h/c. 
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Solution Evaluate the partial derivatives as 


u, = poate Gy — CD exp - (2 = Al 
h h_ oh 


2 Jace - Ct) sep -(2 2 ay 
h hh 


and 


tha atte 


2 
‘i —2ac’ re (#- Vl 4 4a(x = =f c ani -(2 -@) 
he h_ jh h_ jh 


Clearly (9.4) is satisfied by these second derivatives. 
The curves of wu against x are plotted in Figure 9.2, and show a wave initially centred 
at the origin moving with a constant speed c to the right. 


Figure 9.2 up 
Propagating wave tat 
in Example 9.2. i. 


9.2.2 Heat-conduction or diffusion equation 


1 du 
K ot 


2 


=Vu (9.5) 


This equation arises most commonly when heat is transferred from a hot area to a cold 
one by conduction, when the temperature satisfies (9.5). 

In Section 3.6 a full derivation of the equation (9.5) is made. Here we shall invest- 
igate the one-dimensional version in the context of the heat flow along a thin bar. The 
bar is assumed to have a uniform cross-sectional area and an insulated outer surface 
through which no heat is lost. It is also assumed that, at any cross-section x = constant, 
the temperature 7(x, t) is uniform. Consider an element of the bar from x to x + Ax, 
where x is measured along the length of the bar, as illustrated in Figure 9.3. An 
amount of heat O(x, ¢) per unit time per unit area enters the left-hand face and an 
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Figure 9.3 Heat flow 
in an element. 
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x x+Ax 


O(x, ) _§ | --—> Ox + Ax, 1) 


amount O(x + Ax, f) leaves the right-hand face of the element. The net increase per unit 
cross-sectional area in unit time is 


O(x, t) — Ox + Ax, f) 


If c is the specific heat of the bar and p is its density then the amount of heat in the 
element is coTAx. The net increase in heat in the element in unit time is 


ope Ax 


and is equated to the net amount entering. Thus 


ae = O(x, t) — Ox + Ax, A) 


ot 
which in the limit as Ax > 0 gives 
oT __ 20 
cA ot ax ie) 


The Fourier law for the conduction of heat states that the heat transferred across unit 
area is proportional to the temperature gradient. Thus 


at 


Q= Ox 


where k is the thermal conductivity and the minus sign takes into account the fact 
that heat flows from hot to cold. Substitution for O in (9.6) gives the one-dimensional 
heat equation 


2 
ar_ FT 


9.7 
ot ax Cn) 


where k= k/cp is called the thermal diffusivity. 

An entirely similar derivation for the diffusion equation can be made. The only dif- 
ference is that the Fourier law is replaced by Fick’s law that the diffusional flow of a 
material is proportional to the concentration gradient. 

The equations describing more complicated phenomena, such as the time-dependent 
electromagnetic equations or the equations of fluid mechanics, have the same basic 
structure as (9.5), but with additional terms or with coupling to other equations of the 
same type. We certainly need to know how to solve (9.5) before even contemplating 
solving these more complex versions. 

An essential feature of the heat-conduction equation is that, given a long enough 
time and assuming that there are no time varying inputs, the temperature will eventually 
settle down to a steady state. Thus the final solution is independent of time, and hence 
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Example 9.3 


Solution 


Example 9.4 


Solution 


will satisfy Qu/ot = 0 or V?u = 0. The transient behaviour tells how this solution is 
approached from its given starting value. Physically it is reasonable that any initial 
temperature, however complicated, will move to a smooth final solution, and we should 
not expect the severe difficulties with discontinuities that occur with the wave equation. 
Exactly how initial discontinuities are treated in a numerical solution, however, can 
affect the accuracy in the early development of the solution. 


Show that 
T=T,,+(T,,-T..)e0* (x = UD) 


satisfies the one-dimensional heat-conduction equation (9.7), together with the bound- 
ary conditions T > T,, as x 4 « and T= T,, at x = Ut. 


The second term vanishes as x > o, for any fixed ¢, and hence T > T.,. When x = Ut, 
the exponential term is unity, so the 7s cancel and 7 = T,,. Hence the two boundary 
conditions are satisfied. Checking both sides of the heat-conduction equation (9.7), 


1oT_1 U? _uw-unix 
=f..= 5 -T.)~e 
K ot (Tn ) K 


aT 


i U? —U(x-Ut)/K 
Ae 


which are obviously equal, so that the equation is satisfied. 

The example models a block of material being melted at a temperature 7, with the 
melting boundary having constant speed U, and with a steady temperature 7_, at great 
distances. An application of this model would be a heat shield on a re-entry capsule 
ablated by frictional heating. 


Show that the function 


T= i exp(-2) 
\t 4kt 


satisfies the one-dimensional heat-conduction equation (9.7). Plot T against x for various 
times ¢, and comment. 


We first calculate the partial derivatives 


OO tL af ea ee Sl esl Se, 
== 3 EXP + 7 ©XP 


ot 2t 4xt/ \t 4k t 4kt 
2 
of 1 exp) 
ox = \t4Kt 4kt 
and 
OT 


(i a exp(==) Fa 2 exp(==) 
ax 2K” kt) 2Kt” 4xt Akt 
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Figure 9.4 Solution 
of the heat-conduction 
equation starting from 
an initial spike in 
Example 9.4. 


92.3 
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Temperature 
12 
10 
8 
6 
4 
2 

4 -3 2 -l 0) I 2 3 4 

Distance 
mr LH 0.1; rtnan »£=03; see9,L=05; meme Lal. 


It is easily checked that (9.7) is satisfied except at the time t= 0, where T is not properly 
defined. The graph of reduced temperature 7/ (4k) against distance x at various times 
t = L’/4« can be seen in Figure 9.4. Physically, the problem corresponds to a very hot 
weld being applied instantaneously to the bar. The initial temperature ‘spike’ at x = 0 is 
seen to spread out as time progresses, and, as expected from the physical interpretation, 
T tends to zero for all x as the time becomes large. Alternatively the problem describes 
the diffusion of a large pulse of contaminant into a thin tube of fluid. 


Laplace equation 


Vu=0 (9.8) 


The simplest physical interpretation of this equation has already been mentioned, 
namely as the steady-state heat equation. So, for example, the two-dimensional 
Laplace equation 


CHE Ree 

eee 

ox oy 
could represent the steady-state distribution of temperature over a thin rectangular 
plate in the (x, y) plane. 


0 


Heat transfer is well understood intuitively, and good guesses at steady-state solutions 
can usually be made. Perhaps less commonly understood is the case of the electrostatic 
potential in a uniform dielectric, which also satisfies the Laplace equation. Working out 
the electrical behaviour of a capacitor that is charged in a certain manner simply implies 
solving (9.8) subject to appropriate boundary conditions. Possibly the least obvious, 
but extremely important, application of the Laplace equation is in inviscid, irrotational 
fluid mechanics. To a large extent, subsonic aerodynamics is based on (9.8) as an 
approximate model. The lift and drag on an aerofoil in a fluid stream can be evaluated 
accurately from suitable solutions of this equation. It is only close to the aerofoil that 
viscous and rotational effects become important. 

The Laplace equation is a ‘smoother’ in the sense that it irons out peaks and troughs. 
Physically, the steady-state heat-conduction context tells us that if a particular point has 
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a higher temperature than neighbouring points then heat will flow from hot to cold until 
the ‘hot spot’ is eliminated. Thus there are no interior points at which the solution u 
of (9.8) is smaller or larger than all of its neighbours. This result can be confirmed 
mathematically, and establishes that smooth solutions are obtained, see Section 9.7.1. 


Example 9.5 — Show that 
u=x'— 2x*y— 6x’y? + xy? + y* 


satisfies the Laplace equation. 


Solution Differentiating 


u, = 4x° — 6x’y — 12xy* + 2y*, uw, =-2x* — 12x°y + Oxy? + 4y° 


U,, = 12x? — 12xy — 12y?, Uy, = —12x? + 12xy + 12y* 
so clearly 
Ux, + Uy, = 0 


and the two-dimensional Laplace equation is satisfied. 


Example 9.6 — Show that the function 


2 
roft-s6) 
yey 


satisfies the Laplace equation, and sketch the curves y= constant. 


Solution — First calculate the partial derivatives: 


= 2xyUa™ 


x 


—_ ty) 
x+y (x+y) 
_ _2yUa"__ 8x°yUa" 
bs (ety) (x+y) 
__2yUa"_, 4yUa’ _ 8y'Ua’ 
vy (x+y)? +) (+) 
Substituting into (9.8) gives 
Vy= 8yUa" << By(x + y Ua" -0 
(x+y) (x+y) 


and hence the Laplace equation is satisfied. 
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Figure 9.5 
Streamlines for flow 
past a cylinder of 
radius 1, from the 
Laplace equation in 
Example 9.6. 


EJ 


9.2.4 
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Secondly, to sketch the contours, we note that y= 0 on y = 0 and on the circle 
x’ + y? = a’. On keeping y = yy and letting x — +eo, the second term vanishes, so the 
curves tend to y= Uy). Figure 9.5 shows the solution, which corresponds physically to 
the flow of an inviscid, irrotational fluid past a cylinder placed in a uniform stream. 


Computer packages can verify the differentiations and the plotting in any of the 
examples in this section. For instance, the MAPLE instructions 

Sa satay? (=e a (ee 247") )) p 

Glib iE (OSA Se ,52)) @ CLE (osm 7,37) s Salimeulaliny (S23) & 
verify the Laplace equation in Example 9.6. The plotting in Figure 9.5 can be 
achieved from the instructions 


waiela (jo lleie Ss) 2 

gga (LHL / (2° 24572) )) 2 
aacalho=0. 001 ,c=0. cai, cg=i1.5]| 5 

mio Leaieoullore (sao (ei |, tal. 4) } pee ot 0. 2 2, 
scaling=constrained) ; 


Other and related equations 


We discussed in Section 9.1 applications in science and engineering. Many such applica- 
tions are governed by equations that are closely related to the three basic equations 
discussed above. For example, consider the equations of slow, steady, viscous flow in 
two dimensions, which take the form 


dp_ly1, B_1y 
ox RY oy RY” 
Ou , Ov _ 
ae ea 


where wu, v and p are the non-dimensional velocities and pressure, and % is the Reynolds 
number. The system has a familiar look about it, and indeed a little simple manipulation 
gives V*p = 0, so that the pressure satisfies the Laplace equation. If p can be calculated 
then dp/Ox and dp/dy are known, so we have equations of the form 


(9.9) 
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Vu =f(x, y) (9.10) 


This equation is called the Poisson equation, and is clearly closely related to the Laplace 
equation. It can be interpreted physically as steady heat conduction with heat sources 
in the region. A careful study of the solution of the Laplace equation is required before 
either (9.10) or (9.9) can be attacked. In Sections 9.5 and 9.7 some discussions of the 
Poisson equation take place. 

If there is good knowledge about the time behaviour of the wave or diffusion equa- 
tion then we can often obtain important information from them without solving the full 
equations. For instance, if we put a periodic time dependence u =e!“v(x, y, z) into (9.1), 
or if we put an exponentially decaying solution u = e v(x, y, z) into (9.5), then the 
variable v, in both cases, satisfies an equation of the form 


V0 +Av=0 (9.11) 


Equation (9.11) is called the Helmholtz equation, and plays an important role in the 
solution of eigenvalue problems. It is perhaps of relevance that the best studied eigen- 
value equation, the Schrédinger equation, is almost the same, namely 

he 


2 
8mm 


Vu — Vix, y, zu+ Eu=0 


It is a bit more complicated than (9.11), but it forms the basis of quantum mechanics, 
on which whole industries are built. 

So far, all of the equations that we have considered are Jinear, since they have not 
included any quadratic (or higher) terms in u or its derivatives. As soon as we move from 
linear to nonlinear problems, a whole new crop of theoretical and computational difficulties 
arises. Very few such equations can be solved analytically, and devising computational 
schemes is not easy. Even worse, mathematicians cannot always tell whether or not a solu- 
tion even exists. An act of faith is usually made by scientists and engineers that their problem 
is modelled correctly and therefore there must be a mathematical solution reflecting the 
physics. Often the faith is well founded, but modelling is an imperfect art and there are many 
things that can go wrong. It may be thought that nonlinear problems do not occur in practice, 
but this is certainly not the case. For some phenomena, like the behaviour of thermionic 
valves or avalanche semiconductors or pulsed lasers, it is the nonlinearity that produces 
the desired effects. Other situations arise where the nonlinearity of the system may or 
may not be important. For instance, the full steady two-dimensional fluid equations are 


Qu, ,du__@® ly 

le ay 5. Oh u 

dv, ,v_ ® , 1y2 

an ap ay RY Vv (9.12) 
Ou , Ov _ 

a ay 


where u, v, p and & are defined as for (9.9). These equations are nonlinear because of 
the presence of quadratic terms such as u 0u/Ox. It can be seen that (9.12) reduce to (9.9) 
for slow flow when quadratic terms are neglected. While (9.9) would be applicable to 
the flow of molten glass, we would need the full equations (9.12) to look at flow close 
to an aerofoil. Indeed, as % becomes large, the flow becomes turbulent, that is unstable, 
and the applicability of these equations comes into question. 
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Example 9.7 


Solution 
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Arbitrary functions and first-order equations 


In each of the examples in this section, a solution has been given; it has been checked 
that the solution satisfies the appropriate partial differential equation. In no case has the 
boundary condition been part of the specification of the problem, although in several 
cases boundary conditions were checked. In the next sections the boundary conditions 
are given as part of the set-up of the example. This is the natural way that a physical 
problem is specified and it proves to be a much tougher proposition. 

The most significant difference between ordinary and partial differential equations 
is the treatment of the ‘arbitrary constants’. Consider the examples: 


ODE PDE 

Solve the ordinary differential equation Solve the partial differential equation 

dy(t) _ 42 dz(x, t) _ 412 

=3t = 3t 
dt ot 

Integrating gives Integrating gives 

yiaeh+K 2(x, = 0 + f(x) 
where K is an arbitrary constant, since —_ where f(x) is an arbitrary function. 
differentiating y(t) with respect to ¢ Differentiating with respect to ¢ produces 
produces 3/° whatever the value of the 3° for any function f(x) because x is kept 
constant K. constant in the partial differentiation. 


Extending this idea it can be seen that each partial integration introduces an arbitrary 
function into the solution. Sufficient conditions must be given to determine these arbit- 
rary functions. It is not always easy to decide exactly what conditions are required, but 
in subsequent sections an idea will be given for the three classic equations, the wave 
equation, the heat-conduction equation and the Laplace equation. An extended discus- 
sion can be found in Section 9.8. 

Consider for the moment a first-order equation. Such equations are of less interest in 
applications to engineering and science, but there is a comprehensive theory for their 
solution which will illustrate the use of arbitrary functions. 


Find the general solution, u(x, 4), of the partial differential equation 
du , Ou 
—-+=-=0 
ot ox 


and find the particular solution when u(x, 0) = x’. 


Change the variables z = x — t and T= ¢ and use the chain rule to evaluate the terms in 
the equation 


du _ Qudz, duaT __du , du 
ot ozot oToat oz oT 


Qu _ Oude, dua _ du 
ox ozox OTOx a 
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Figure 9.6 Surface 
z= f(x, y) showing the 
tangent, normal and 
characteristic curve C. 


Putting these differentials into the equation 


_ du, du _ du 
ot ox oT 
Thus u(z, 7) can be deduced as 

u(z, T) = f(z), where fis an arbitrary function 
Reverting to the original variables 


u(x, t) = f(x a t) 
and a general solution of the partial differential equation has been obtained. 
For the particular solution with initial conditions written in parametric form, x = s, 
t= 0, u=s", it is easily deduced that s” = f(s) and hence 


u(x, t) = (x - 


The solution of quasi-linear first-order equations with two variables, x and y, is com- 
paratively straightforward 


P(x, y, jg +Q(x, y, ne = R(x, y, Z) (9.13) 


Provided P, Q and R are ‘well behaved’ a method of solution can be deduced, although 
the resulting integrals cannot always be obtained explicitly. Extension to many variable 
problems is similar, but the geometrical interpretation is more difficult. 

In Section 3.2.1 it was seen that the function z = f(x, y), illustrated in Figure 9.6, has a 
normal (0z/dx, dz/Ay, —1) at a typical point M, having coordinates (x, y, z). Equation (9.13) 
says that the normal to the surface is perpendicular to the vector (P, QO, R) at the point 
M and thus (P, Q, R) must lie in the tangent plane. Now examine the curve C in the surface 
starting at the point A and moving along C in a direction that is always parallel to (P, 
Q, R) at the current point. The direction therefore remains perpendicular to the normal 
(0z/Ox, dz/Ay, —1) at all points and must move in a tangential direction to the surface; 
such curves are called characteristic curves. The point must remain in the surface and 
this tangential direction (dx, dy, dz) must therefore be parallel to (P, QO, R) so that 


dx _dy_ dz 
P OR 


(9.14) 


normal (z,, Z,, —1) 


tangent (P, Q, R) 
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Solution 


9.2 GENERAL DISCUSSION 737 


Starting from (9.13) we have shown that z(x, vy) can be obtained from the two ordinary 
differential equations (9.14). If we start from (9.14) we know that the normal direction 
(0z/Ox, dz/dy, —1) is perpendicular to the tangent vector (dx, dy, dz) and hence perpen- 
dicular to (P, Q, R) so 


PUx, ¥, 2) +0(x y, nz — R(x, y,z) =0 


and (9.13) is satisfied. 

From a particular starting point, x =a, y=b, z=c, the solution of (9.13) is obtained as 
the characteristic curve obtained from the solution of the ordinary differential equations 
(9.14). Usually there is a starting curve; then essentially the process calculates the charac- 
teristic curve from each point of the starting curve and the solution surface is generated. 

To illustrate the method return to Example 9.7 when the equations (9.14) become 


dt_ dx _ du 
1 1 0 

using the variables given. The two ordinary differential equations are 
ee 1 with solution x-—t=A 
dt 
du 


—=0 withsolution uw=B8 
dt 


The constants A and B are arbitrary and are determined from a given initial data point. 
Usually the initial data is given on a curve t= 0, x = f(s), y = g(s), so for each s there are 
arbitrary constants A and B; in this case, the constants depend ons, that is A(s) and B(s). 
In the current example (9.7) the initial data is ¢= 0, x = 8, u = s* giving 


S=A so x-t=s 
’=B so u=s’ 


Eliminating s gives u = (x — f) as deduced earlier. A further example shows how the 
method is applied. 


Solve the equation 


for z(x, y) given that z= f(s) when x = s andy=1--s. 


The two ordinary equations obtained from (9.14) are 


dx_dy anq Ox _ dz (9.15a,b) 
x y x xy 


Solving (9.15a) gives In x = In y + C which reduces to x = Ay. Putting this result into 
(9.15b) gives 


xdx = Adz 


which on solving gives 
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1H Az+B 


To obtain the arbitrary constants A and B we insert the initial conditions 


Se a (9.16a) 
y I-s 

lo x ie S 

Se ee a 9.16b 
pt - S25 58-7 Ae) (9.16b) 


Thus A and B have been obtained in terms of s. From (9.16a) we get 


*X=-S— andhence s =—~ 
y I-s x+y 


So that (9.16b) becomes 


lox = J 2/(— ) 
ax 22 = =| 2 | -t7(/ 
2 y 2\x +y y \xt+y 


Rearranging, z is then calculated as 
_ 1y-Ly 4 (5) 
2 2(x +y) x +y 


The solution can be checked by substitution into the original differential equation. 


(2) Because there is a comprehensive theory of the solution of some classes of first- 
order partial differential equations computer packages can be used to solve these 
equations with comparative ease. The MAPLE instructions 

with(PDEtools) : 
IDS Hse Clavieit (sz (Se,57) 432) ae seecluitit (2 (se,97) py) = Se aws 
pdsolve (PDE) ; 


produce the general solution in Example 9.8. 


<A 

xy 

Figure 9.7 Draining A practical example of first-order equations involves the draining of liquid from a vessel, 
of liquid down the side a procedure common to many industrial processes. The thickness of the liquid layer is 
of a vessel. required as time progresses. 


Example 9.9 A thin layer of liquid drains down the side of a vessel, as illustrated in Figure 9.7. From 
the theory of thin layers, the equation for the fluid motion is given by 
oh 20h 
=+ah'— =0 
at ax 


where A(x, f) is the thickness of the layer and a is a constant that depends on the viscosity, 
density and the gravity constant. Find the solution for h(x, f) given the initial condition 
h(x, 0) = ax where @ is a constant. 


Solution The ordinary differential equations (9.14) are 


dt _ dx _ dh 
1 gp 90 
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Clearly h = C, a constant, solves one of the equations and the other is 


“ =ah’=aC  withsolution x=atC?+K 


where C and K are arbitrary constants. Thus the solution of the equation is 


(ee 
K=x-ath 


Using MAPLE to try for a solution, the instructions 


with(PDEtools) : 
Glic@iatin s=Glil ise (ini (Se, E)) , 1) shel la (Re, E)) “Aivela wie (im (Fe, 12) p22) F 
pdsolve (drain); 


give / as a solution of the equation 
fh) =x- ath’ 


Note that the package has combined the arbitrary constants C and K into one arbitrary 
function f, determined by the initial data. 

Clearly the package can go no further without the specification of the initial condi- 
tion. Putting in the conditions x = s, t= 0, h = as gives fas 


floys)=s or f(p)= (2) 


The function / can now be calculated as 


ia, ) =o" | . 


l+adt 


1 
and shows that, for large 1, the layer thins at a rate proportional to ft * . The solution can 
be checked by direct substitution into the original equations. A plot of / against x at 
successive times or a three-dimensional plot of h(x, ft) using PDEplot in MAPLE can 
be used to illustrate the solution. 


The MATLAB instructions that produce the surface shape, //a, at successive times 
act = 0,2,4,8 are 

y=0:0.1:4; 

Se [somes (vy) ° , Senete (7/3) © , Sienete 7 /S) * , Senete (W/D)) * | g 

folloie (2¢, 57) 


Further applications of first-order equations occur in the study of the time evolution of 
the probability distribution of the position of a particle, for instance in Brownian motion. 
The equation is 


AED 4 Ip, yf, y= Aix, Of, 0) 
Ot ox Ox 


where D, and D; are drift and diffusion coefficients. If diffusion can be neglected then 
the equation is just a first-order partial differential equation. 
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In gas dynamics and also in traffic flow problems a similar equation, the Burgers’ 
equation, can be shown to apply 


In the two situations wu is the gas velocity or the traffic density and v is a viscosity 
coefficient. For the inviscid case, again the equation reduces to a first-order partial dif- 
ferential equation. The derivation of these equations is lengthy and beyond the scope of 
this text but can be found in specialist books. 

The solution of inviscid Burgers’ equation is obtained from 


dt _dx_ du 
1 u 0 

The two equations give one obvious solution uw = A and the second is 
dx 


—=u=A andhence x=4t+B 


Taking initial conditions t= 0, wu = V(s) and x = s we obtain 


A=u=Vs) 


B=x-ut=s 


Eliminating s gives the solution for u(x, 4) in implicit form 


Vix —ut)=u 


9.2.6 Exercises 


Find the possible values of a and 5 in the 
expression 


u = cosat sin bx 


such that it satisfies the wave equation 


Lou _Fu 

© Ot ax 
Taking 

u=f(x+ at) 


where fis any function, find the values of o that 
will ensure that w satisfies the wave equation 
1au_ ou 


2 


c Ot ax 
Verify that the function 

u(x, y) =x*- 6x’y + y" 
satisfies the Laplace equation. 


The function z(r, t) depends only on the radial 
distance in spherical polar coordinates and on 
the time. The wave equation in this coordinate 
system is 


dz ,20z_1dz 

—45 =-—-= 

or ror or 
Show that z(r, t) = 1! cos (r — ct) satisfies the 
equation (r # 0). 


Find all the possible solutions of the heat- 
conduction equation 


lou _ Fu 
Kot ox 
of the form 


u(x, t) = e“V(x) 
Find the values of the constant n for which 
V=r"(3cos’@- 1) 


satisfies the Laplace equation (in spherical polar 
coordinates and independent of @) 


2 (re) 1 (si al 
=|(7 = | é— |=0 
a o) snede. a0 
for all values of the variables r and 0. 


Show that u = e“ cos mx cos nt is a solution of the 
equation 
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c Tu = Tu + 2424 
ox” ar ot 
provided that the constants 4, m, n and c are related 
by the equation n* + k? = c?m’. 
If V=x* + axy’, where a is a constant, show that 


av, av _ 
ox ~ oy 


Find the value of a if V is to satisfy the equation 


x=+y 3V 


rh ae 
ox” oy 


0 


Taking this value of a, show that if uw =7°V, where 
=x’? +y”’, then 
2 2 
gu, eu ony 
2 ) 2 


ox" y 
The telegraph equation has the form 


Fo _1 4 22) 

dx c\ Or at 

where c’ is the speed of light and & is usually 

small. Given that ®(x, f) is a solution of the wave 

equation 
a® 


_1d@ 
axe ar 


show that @(x, ft) = ®(x, t)e“” is a solution of the 
telegraph equation, if terms of order k? can be 
neglected. 


The transmission-line equations represent the flow 
of current along a long, leaky wire such as a 
transatlantic cable. The equations take the form 


al _ dv 
mere ey 
Ov _ 4,2 
ae aS, 


where g, c, r and L are constants and / and v are the 
current and voltage respectively. 


(a) Show that when r= g=0, the equations reduce 
to the wave equation. 

(b) Show that when L = 0, the equations reduce 
to a heat-conduction equation with a 
forcing term. Write W = ve® to reduce 
to the normal form of the equation. 
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(c) Puta= 5 (r/L + g/c) and then w = ve”. Show 
that when rc = gL, w satisfies the wave 
equation. 


Show that if fis a function of x only then 
u = f(x) sin(ay + b) 


where a and + are constants, is a solution of the 
partial differential equation 


Ju_ fu ou 


==> -2a— 
Oy ax ox 


provided that f(x) satisfies the ordinary differential 
equation 


ff qd ap =0 
dx dx 


Hence show that 
u=(A + Bx)e“ sin (ay + b) 
where A and B are arbitrary constants, is a solution 


of the partial differential equation. 


Show that f(x, y) =x’ + g(x/y) satisfies the partial 
differential equation 


Of 4D ayy? 
ae 4x"y 


for any arbitrary function g. It is given that f=? on 
the line with parametric equationx =1-—t,.y=¢; 
find the function g. 

Show that the partial differential equation 


Fu, du_ 
oxov ax 


has the general solution 
u(x, y) = e°[ f(x) + g(y)] 


where f and g are arbitrary functions. 


0 


Find the general solution for u(x, y) in the equation 
(check using MAPLE) 


ee pee 
aoe oy 


Show that the solution that satisfies the conditions 
u=s’,x=s,y=1 takes the form 


=(x+y)u 


2.2 
c= 
xy-x+y 
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Solution of the wave equation 


9.3.1 


In this section we consider methods of solving the wave equation introduced in 
Section 9.2.1. 


D’Alembert solution and characteristics 


A classical solution of the one-dimensional wave equation 


du _ du 


9.4 
Cot ax ee 


is obtained by changing the axes to reduce the equation to a particularly simple form. 
Let 


r=x-+ct, s=x-ct 


Then, using the chain rule procedure for transformation of coordinates (see Section 
3.1.1), 


Uyy = Upp + QU, + Use 
Uy = CU — ZU, + Ugg) 

so that the wave equation (9.4) becomes 
4c’u,, = 0 


This equation can now be integrated once with respect to s to give 


_ ou _ yx, 
up = 3 = Or) 


where @ is an arbitrary function of r. Now, integrating with respect to r, we obtain 
u=f(r) + g(s) 

which, on substituting for r and s, gives the solution of the wave equation (9.4) as 
u=f(x + ct) + g(x - ct) (9.17) 


where fand g are arbitrary functions and f is just the integral of the arbitrary function 6. 

The solution (9.17) is one of the few cases where the general solution of a partial 
differential equation can be found. However, finding the precise form of the arbitrary 
functions fand g that satisfy given initial data is not always easy. The initial conditions 
must give just enough information to evaluate fand g, which are functions of the single 
variables r = x + ct and s = x — ct respectively. 

In Example 9.2 we have already seen a simple example of a wave of this type. We 
first deduced that a function of x — ct satisfied the wave equation, and then showed in 
Figure 9.2 that it represented a wave travelling in the x direction with velocity c. 

The next example is similar. 
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Example 9.10 


Solution 


Check that uw = 1/[1 + (x + cf)’] satisfies the wave equation (9.4) and show that it 
represents a travelling wave in the —x direction. 


Differentiating partially with respect to x and ¢ 


op PEEL) : _2[-143(x+cr)’] 
* Tl+(xtetY 7 “1+ (xtetP° 

pies —2c(x+ct) Je -14+3(x+ct) 
" t14(x+et/]" *  a@een Tr 


and the wave equation is satisfied. Plots of the function wu against x for various values 
of ct are shown in Figure 9.8. The same curve can be seen to be just translated to the left. 


—s— c=0 
—e— c= 
—A— ct=2 
=—B— a3 
—e— ch=5. 


2 -1 0 
x 


Figure 9.8 Solution to Example 9.10 showing wu against x for various values of ct. 


Example 9.11 


In Example 9.11 we attempt the more difficult task of fitting initial conditions to the 
solution. 


Solve the wave equation (9.4) subject to the conditions 
(a) zero initial velocity, du(x, 0)/dt = 0 for all x, and 
(b) an initial displacement given by 
lex (= 7= 1) 
u(x, 0)=F(Qx)=j)14+x (-1 <x <0) 


0 otherwise 
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Solution This example corresponds physically to an infinite string initially at rest, and displaced 
as in Figure 9.9, which is then released. 
u From (9.17) we have a solution of the wave equation as 


u=f(x + ct) + g(x — ct) 
We now fit the given boundary data. Condition (a) gives 


a . a 0=cf'(x)-cg’(x) forall x 


Figure 9.9 Initial 


displacement in so that 
Example 9.11. f(x) — g(x) = K = an arbitrary constant 
and thus 


u=f(x+ct)+f(x—-—ct)-—K 
Similarly, condition (b) gives 
F(x) = 2f(x) — K 
so that 
u=}F(x+ct)+}$F(x- ct) (9.18) 


We now have the solution to the equation in terms of the function F defined in condition 
(b). (Note that the same is true for any function F.) 

The solution is plotted in Figure 9.10 as wu against x for given times. It may be 
observed from this example that we have two travelling waves, one propagating to the 
right and one to the left. The initial shape is propagated exactly, except for a factor of 
two, and the shape discontinuities are not smoothed out, as noted in Section 9.2.1. 


Figure 9.10 Solution 
to Example 9.11 
showing two waves 
propagating in the 

+x and —x directions 
with velocity c. 


The analysis in Example 9.11 can be extended to solve the wave equation subject to 
the general conditions 


(a) an initial velocity, du(x, 0)/dt = G(x), and 
(b) an initial displacement, u(x, 0) = F(x) for all x. 
Condition (a) gives, from (9.17), 
G(x) = cf f’"@) - 2@)] 
so that 


x 


cLf (x) - g(x)] = | G(x)dx+Ke 


Condition (b) gives 
f(x) + g(x) = FX) 
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Figure 9.11 Solution 
to Example 9.11 in 
(x, t, u) space. 
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and we can solve for f(x) and g(x) as 
f(x) = LF(x) + > | G(x) dx +1K 
(63 
0 


g(x) = LF(x) - > | G(x) dx- 1K 
Cc 
0 
The solution thus becomes 
[F(x + ct) +F(x-ct)]+ > | G(z)dz (9.19) 
Cc 


x-ct 


w=! 
which is commonly called the d’Alembert solution. As in Examples 9.2, 9.10 and 
9.11, it gives rise to waves propagating in the +x and —x directions. 

As mentioned in Section 9.1, a difficulty is to illustrate the solution of a partial differential 
equation in a simple way. Figure 9.10 is a ‘snapshot’ at a particular time ¢, and if we wish 
to look at the solution over all (x, f) then we have to draw w as a function of the two variables 
x and t. We can draw the solution to Example 9.11 in a three-dimensional diagram as in 
Figure 9.11, but for any higher-dimensional problem such a diagram is clearly impossible. 
The ‘snapshot’ in Figure 9.10 corresponds to a plane slice parallel to the (uw, x) plane. 

The d’Alembert solution, which reduces to an integral along the boundary, does not 
have any simple extension other than for the x axis. In Section 9.7.2 the Green’s function 
is introduced and it can be interpreted as an extension since it involves integrals round 
the boundary of a general region. However, the calculation of the Green’s function is a 
tough proposition for any but the simplest regions. Following from the idea of the 
d’Alembert solution, characteristics (which will be studied in the next few paragraphs) 
can be used to extend the range of boundaries that can be dealt with. 

The idea of using an (x, #) plane is a very useful one for the wave equation, since the 
solution 


u = f(x + ct) + g(x — cf) 


gives a representation by characteristics. If we plot the lines x + ct = constant and 
x — ct = constant as in Figure 9.12 then we see that the line AP has equation x — ct = xy 
and the line BP has equation x + ct = x,. Thus 


on the whole of AP g(x — ct) = g(x) 
on the whole of BP f(x + ct) = f(x,) 


www.2Ofile.org 


746 PARTIAL DIFFERENTIAL EQUATIONS 


Figure 9.12 
Characteristics 

x + ct = constant and 
x — ct = constant. 


Thus g takes a constant value on AP and f takes a constant value on BP. If we can 
calculate fand g on the initial line t= 0 then we know the value of u at P, namely 


u(P) = f(x) + 8%) (9.20) 


Since P is an arbitrary point, the solution at any point would be known. The essential 
problem is to calculate f(x) and g(x) on the line ¢= 0. 
Typical conditions on t = 0 are 


(a) u(x, 0) = F(x), and 
(b) du(x, 0)/dt = G(x), 


which specify the initial position and velocity of the system. Now 


ot + a = cf’(x + ct) + cg’(x — ct) + cf’"(x + ct) — cg’(x — ct) = 2cf"(x + ct) 
i 
and similarly 
Ou _ BU _ yore - 
oa. a 2cg"(x — ct) 


On t = 0 we know that du/dx = F’(x) and du/dt = G(x), so we can deduce that 
cF’ (x) + G(x) = 2cf"(x) 
cF’ (x) — G(x) = 2ce’(x) 
Since F and G are given, we can compute 
f(x) = SIF’) + G@)/c] 
8 (x) = 51’) — G@)/e] 


and hence f(x) and g(x) can be computed by straightforward integration. 

This method is essentially the same as the d’Alembert method, but it concentrates 
on calculating f(x) and g(x) on the initial line and then constructing the solution at P by 
the characteristics AP and BP. The method gives great insight into the behaviour of the 
solution of such equations, but it is not an easy technique to use in practice. Perhaps the 
best that can be obtained from characteristics is an idea of how the solution depends on 
the initial data. In Figure 9.13 the characteristics emanating from the initial line are 
drawn. To evaluate the solution at P, we must have information on the section of the 
initial line AB, and the rest of the initial line is irrelevant to the solution at P. This is 
called the domain of dependence. The section of the initial line AB has a domain of 
influence determined by the characteristics through the points A and B. The data on AB 
cannot influence the solution outside the shaded region in Figure 9.13(b). 
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Figure 9.13 
Characteristics 
showing (a) the 
domain of dependence 
and (b) the domain of 
influence. 


Domain of influence 


A 


Domain of 


Ly, : 
ay ,. 
dependence 4 Ling 


(a) (b) 


Example 9.12 Use characteristics to compute the solution of the one-dimensional wave equation (9.4), 
with speed c = 1, given (a) the initial conditions that u = V(x) and du/dt = 0, for x > 0 
and ¢ = 0, and (b) the boundary condition that w = 0 at x = 0 and ¢ > 0. Describe the 
solutions in the particular cases 


x 


ae al 


(i) Vix)=1 and (ii) Vix) = 


Solution The characteristics are plotted in Figure 9.14. It can be seen that for x > ¢, at a typical 
point P two characteristics emanating from the initial line, t= 0, meet and the solution 
can be computed at P from data on the initial line. However, for x < ¢, at a typical point 
Q the characteristics emanating from the boundary, x = 0, are required. These observa- 
tions will be borne out in the mathematical computations. 

For the region x > ¢ the characteristic analysis described in the previous analysis can 
be followed. It was shown that fand g in the solution 


u=f(x—f)+g(x+P) (9.21) 


Figure 9.14 3 
Characteristics for 
Example 9.12. 


oO 
S 
vn 
= 


1.5 2 2.5 


w 
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can be calculated, taking G(x) = 0, as 
f'@=57V'@) and g’@)=7V'@) 
Integrating, and putting the arbitrary constant to be zero, gives 
f(x) =$V(x) and g(x)=4}V(x) forx>0 
and hence the solution 
u(x, )=43[Viaat)+V(x-f] for x>t (9.22) 


For the region x < f, (9.21) requires f(z) at negative values of z. The function is not yet 
known for negative values and must be determined by the other condition (b) on x = 0. 
The condition u = 0 at x = 0 and t > 0 implies in (9.21) 


0=f(-) + sO 
and hence for a general variable z 
f(-2) =-g@)=-3V(z) for z>0 
Using this result, (9.21) gives the required solution 
u(x, t)=43[Viaet+a—-Vit-x)] for x<t (9.23) 


The complete solution for all x > 0 and ¢ > 0 is now known from (9.22) and (9.23). 


Case (i) 


In this case V(x) = 1 so (9.22) gives u = 1, in the shaded region of Figure 9.14, and 
(9.23) gives u = 0, in the unshaded region of Figure 9.14. Thus 


1 for x>t 
u(x, t) = 
0 for x<t 


Note that the discontinuity in the boundary data at x = 0, t= 0 is propagated along the 
characteristic x = ¢. 
Case (ii) 


Putting the function V(x) = x/(x* + 1) into (9.22) and (9.23) gives the solution 


1 GT xX+t 


3 5 2 a a for x>t 
(x-ty +1 “(x4+t) +1 

u(x, t) = 
L_fax 1ST tyr x <i 
(t-x) +1 “(x+r) 41 


The boundary data are now smooth so the function u(x, f) remains smooth as illustrated 
for three cases in Figure 9.15. 

The basic physical problem described in this example is a very long string held at 
one end and initially at rest. The string is then displaced at ¢ = 0 in the shape of the 
function V(x) and released. 
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Figure 9.15 
Smooth solutions 
for Example 9.12, 
Case (11); string 
displacements at 
various times. 


Example 9.13 


Solution 
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In more complicated problems, the evaluation of the arbitrary functions f and g in 


equation (9.17) and the use of characteristics is no longer straightforward. We do not 
have a d’Alembert type of solution; a great deal of thought and care is needed. 


The idea of characteristics can be applied to more general classes of second-order 


partial differential equations. Example 9.13 illustrates a case of a constant-coefficient 
equation. 


Find the characteristics of the equation 


O=u,, + 2u,,+ 20uU, 


Study the case when a = : and the solution satisfies the boundary conditions 


(a) 
(b) 


(c) 
(d) 


du(x, O)/Ot=0 forx =0 
1 (0<x<1) 
0 (x21) 


u(0,t)=0 forall t 
du(0, t)/dx=0 for all ¢ 


u(x, 0) = F(x) = 


Since the coefficients of the equation are constants, we know that the characteristics are 
straight lines, so we look for solutions of the form 


u = u(x + at) 


Putting z = x + at and writing uw’ = du/dz and so on, we obtain 


O=u,, + 2u,,+ 20u,= (1 + 2a + 2a’ au” 


Hence for a solution we require 


or 


1+2a+2a°a=0 


.=1+ JC =2a) 
(( — 


2a 
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Figure 9.16 
Characteristic solution 
of Example 9.13. 

The solution u takes 
the constant values 
shown in the six 
regions of the first 
quadrant. 


Ifa> then the two values of a (a, and a, say) are complex, and the characteristics 
x + a,t = constant and x + a,t = constant do not make sense in the real plane. 

If a= } then both roots give a = 1, and we only have a single characteristic x + t= 
constant, which is not useful for further computation. 

For the case ~@ < ‘, we find two real values for a and two sets of characteristics. 

It is precisely for this reason that characteristics serve no useful purpose for the heat- 
conduction or Laplace equations. A further discussion can be found in Section 9.8 after 
the formal classification of equations has been completed. 

Take the case a= 2; then we obtain a, =—2 and a, = —?, so the solution has the form 


u=f(x — 2t)+ ga- =t) 


where f and g are arbitrary functions and the characteristics are the straight lines x — 2¢ 
= constant and x — = ¢= constant. 

The boundary conditions given in the problem are a little more complicated than in 
the d’Alembert solution. Conditions (a) and (b) give 


— u(x, 0) _ a9) 297 
On FOOL Cog 
F(x) = u(x, 0) =f(x)+g(x) 


Taking f(0) = g(0) = 0, we can integrate the first of these expressions and then solve for 
J (x) and g(x) on the line t= 0 as 


SQ) =-7 FQ), g@)=7F@) > 0) 


Conditions (c) and (d) say that u(0, 4) = 0, and du(0, t)/ox = 0. Thus on the line x = 0 
we deduce 


fQ=s2Z=0 E<9) 


We can now construct the solution by characteristics. Figure 9.16 illustrates this 
solution. Because f(x) and g(x) are constant along the respective characteristics, we 
deduce u(A) = 0, u(B) = —}, u(C) = 1, uD) = 0, u(E) = 3, u(F) = 0 at typical points in 
the six regions that divide up the first quadrant of the (x, f) plane. 


e) 0.5 1.0 1.5 2.0 2.5 3.0 3.5 
Distance 
, ae = 26 —-—,3x=2t+3; —--—,x=2t; ----,x=2t+1. 
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For non-constant-coefficient equations the characteristics are not usually straight lines, 
which causes computational difficulties. In particular, there are some fundamental prob- 
lems when characteristics of the same family intersect. The solution loses its uniqueness, 
and ‘shocks’ can be generated. The classical wave equation (9.4) will propagate these 
shocks, but it requires ‘curved characteristics’ to generate them. 


Separated solutions 


A method of considerable importance is the method of separation of variables. The 
basis of the method is to attempt to look for solutions u(x, y) of a partial differential 
equation as a product of functions of single variables 


u(x, y) = X(x)¥(y) 


The advantage of this approach is that it is sometimes possible to find X and Y as solu- 
tions of ordinary differential equations. These are very much easier to solve than partial 
differential equations, and it may be possible to build up solutions of the full equation 
in terms of the solutions for X and Y. A simple example illustrates the general strategy. 
Suppose that we wish to solve 


Ou , Ou 
—+—=0 
ox oy 
Then we should write u = X(x)Y(y) and substitute 
yeiet 2p) oe 2 
dx dy X dx Ydy 


Note that the partial differentials become ordinary differentials, since the functions are 
just functions of a single variable. Now 


LHS = ae a function of x only 
X dx 
RHS = eee a function of y only 
Ydy 


Since LHS = RHS for all x and y, the only way that this can be achieved is for each side 
to be a constant. We thus have two ordinary differential equations 


1d¥_, _lay_, 
Xdx ° = Ydy 
These equations can be solved easily as 
x=Be™ FaCe” 
and thus the solution of the original partial differential equation is 
u(x, y) = X(e)Vy) = Ae 


where A = BC. The constants 4 and A are arbitrary. The crucial question is whether the 
boundary conditions imposed by the problem can be satisfied by a sum of solutions of 
this type. 

The method of separation of variables can be a very powerful technique, and we 
shall see it used on all three of the basic partial differential equations. It should be 
noted, however, that all equations do not have separated solutions, see Example 9.2, 
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Example 9.14 


and even when they can be obtained it is not always possible to satisfy the boundary 
conditions with such solutions. 

In the case of the heat-conduction equation and the wave equation, the form of one 
of the functions in the separated solution is dictated by the physics of the problem. We 
shall see that the separation technique becomes a little simpler when such physical 
arguments are used. However, for the Laplace equation there is no help from the 
physics, so the method just described needs to be applied. 

In most wave equation problems we are looking for either a travelling-wave solution 
as in Section 9.3.1 or for periodic solutions, as a result of plucking a violin string for 
instance. It therefore seems natural to look for specific solutions that have periodicity 
built into them. These will not be general solutions, but they will be seen to be useful 
for a whole class of problems. The essential mathematical simplicity of the method 
comes from only having to solve ordinary differential equations. 

The above argument suggests that we seek solutions of the wave equation 


of the form either 

u= sin (cAf)v(x) (9.24a) 
or 

u = cos (cAt)u(x) (9.24b) 


both of which when substituted into (9.4) give the ordinary differential equation 


2 
ae =-Wv 
dx 


This is a simple harmonic equation with solutions v = sinAx or v= cos Ax. We can thus 
build up a general solution of (9.4) from linear multiples of the four basic solutions 


u, = cosActsinAx (9.25a) 
Uy = cos Act cos Ax (9.25b) 
u, = sinActsinAx (9.25¢) 
Uu, = sinAct cos Ax (9.25d) 


and try to satisfy the boundary conditions using appropriate linear combinations of 
solutions of this type. We saw an example of such a solution in Example 9.1. 


Solve the wave equation (9.4) for the vibration of a string stretched between the points 
x = 0 and x =/ and subject to the boundary conditions 


(a) u(0,t)=0 (t=O) (fixed at the end x = 0); 
(b) u(l,)=0 (t=O) (fixed at the endx=/); 
(c) du(x,0)/0Ot=0 (0x S</) (with zero initial velocity); 


(d) u(x, 0)=F(x) (given initial displacement). 
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Consider the two cases 
(i) F(x) =sin(mx/1) + ‘sin (31x /1) 


x (0 <x <}I) 
(ii) y= | 


I-x (Ql<x<J 


Clearly, we are solving the problem of a stretched string, held at its ends x = 0 and x =/ 
and released from rest. 

By inspection, we see that the solutions (9.25b, d) cannot satisfy condition (a). We 
see that condition (b) is satisfied by the solutions (9.25a, c), provided that 


sindAl=0, or Al=nn (n=1,2,3,...) 


It may be noted that only specific values of A in (9.25) give permissible solutions. Thus 
the string can only vibrate with given frequencies, nc/2/. The solution (9.25) appropriate 
to this problem takes the form either 


u = COS (22m) sin (2) (9.26a) 
or 

“= sin{ 2) sin{ “= (9.26b) 
(n=1,2,3,...). To satisfy condition (c) for all x, we must choose the solution (9.26a) 


and omit (9.26b). Clearly, it is not possible to satisfy the initial condition (d) with 
(9.26a). However, because the wave equation is linear, any sum of such solutions is also 
a solution. Thus we build up a solution 


u= 5b, cos ( MCR sin( (9.27) 


Case (i) 


The initial condition (d) for u(x, 0) gives 


Dy b, sin (=) = sin (=) + isin (=) 
n=1 


and the values of b,, can be evaluated by inspection as 


b, =1,b,=0,b;=1,b,=b,=...=0 


The full solution is therefore 


_ Tct\ . (mx) 4 3mct\ . (3mx 
u = COS — sin <3 + 7,Cos — sin a 


The solution is illustrated in Figure 9.17. 
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Figure 9.17 
Sketch of the solution 
to Example 9.14 (i) 


with c=! and /= 1. 


Case (ii) 
The condition (d) for u(x, 0) simply gives 


(0<x<4) 
b nx = 2 
p vin IE ae \ -x (l<x<J 


n=1 


and thus to determine b,, we must find the Fourier sine series expansion of the function 


f(x) over the finite interval 0 < x < /. We have from (7.33) that 


I 
ae | fx) sin( 2 Jax 
/ / 
0 
1/2 I 
= xsin (“ax (/- x) sin(“™) ax 
/ / / / 
0 1/2 
= sin(knn) (n= 1,2,3,..-) 


The complete solution of the wave equation in this case is therefore 


u(x, t)= ais J s sin (}nn) cos( WR sin(“=*) (9.28) 
1: c= ‘n° I l 


cht Tx) | 3cmt) . (3mx 
u(x, t)=— ¥ | cos( sin - 5 cos| —— |sin| ——= 
T I l l I 


or 


The complete solution to Example 9.14 Case (ii) gives some very useful information. 
We see that all the even ‘harmonics’ have disappeared from the solution and the ampli- 
tudes of the harmonics decrease like 1/n’. A beautiful theory of musical instruments 
can be built up from such solutions. We see that for different instruments different 
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Figure 9.18 Solution 
of Example 9.14 (11) 
with c= + and/=1. 


Example 9.15 


Solution 
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Displacement 


oO 0.2 0.4 0.6 0.8 1.0 1.2 
Distance 
—,/=0; meme, f= 0,2; exes, f=0.4; 


i fedG: wm F208; “mento, 


harmonics are important and have different amplitudes. It is this that gives an instru- 
ment its characteristic sound. 

A sensible question that we can ask is whether we can use the sum of the series in 
(9.28) to plot wu. In Chapter 7 we saw that convergence is not simple to prove, but the 
Dirichlet conditions in Theorem 7.2 are sufficient to ensure pointwise convergence. 
At discontinuities in particular, Fourier series can be very slow to converge, so that, 
although (9.28) is a complete solution, does it provide us with any useful information? 
In the present case there is no particular problem, but the general comment should be 
noted. The solution (9.28) is plotted in Figure 9.18 with 7= 1 and c= - We note that 
even with 10 terms u(0.5, 0) = 0.4899 instead of the correct value 0.5, so there is a 2% 
error in the calculated value. Perhaps the most pertinent comment that we can make is 
that a good number of terms in the series are required to obtain a solution, and exact 
solutions may not be as useful as we might expect. 

Separated solutions depend on judicious use of the known solutions (9.25) of the 
wave equation to fit the boundary conditions. Although it is not always possible to solve 
any particular problem using separated solutions, the idea is sufficiently straightforward 
that it is always worth a try. The extension to other equations and coordinate systems 
is possible. The use of other orthogonal functions was introduced in Section 7.7 and 
some of these will be discussed in Sections 9.4 and 9.5. 


Solve the wave equation (9.4) for vibrations in an organ pipe subject to the boundary 
conditions 


(a) u(0,t)=0 (t=0) (the end x =0 is closed); 
(b) dul, i\/ox=0 (t=O) (the end x =/ is open); 
(c) ux,0)=0 (OSxS/) (the pipe is initially undisturbed); 


(d) du(x,0V/o0t=v=constant (0<x</) (the pipe is given an initial uniform blow). 


From the solution (9.25), we deduce from condition (a) that solutions (9.25b, d) must be 
omitted, and similarly from condition (c) that solution (9.25a) is not useful. We are left with 
the solution (9.25c) to satisfy the boundary condition (b). This can only be satisfied if 


cosAl=0, or Al=(n+})n (n=0,1,2,...) 
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Thus we obtain solutions of the form 


+4)tct +1 
u= bain on Ot (n=0,1,2,...) 


giving a general solution 


= we Dibusi sn) += pet in al 


The condition (d) gives 


v5 


n=0 


we one =)Tc . aad 
ie 


which, on using (7.33) to obtain the coefficients of the Fourier sine series expansion of 


the constant v over the finite interval 0 < x < /, gives 


2v i _ 8lv 1 
~ (n+ in (n+! Yme e(2nt+1) 


n 


Our complete solution of the wave equation is therefore 


= ae Seale! \n ‘| sn (ne \n 


or, 
8luy (=) : (2) ioe (222) : (=) 
u= >| sin sin| == | + ein| ——— |sin| =—— 
Tc 21 21 21 21 
. (S5tct\) . (5x 
+ 4 sin( SBE) sin( SE) + eve, 


It would be instructive to compute this solution and compare it with Figure 9.18, which 
corresponds to the solution of Example 9.14. 


9.3.3 Laplace transform solution 


For linear problems that are time varying from 0 to ©, as in the case of the wave 
equation, Laplace transforms provide a formal method of solution. The only difficulty 
is whether the final inversion can be performed. 
First we obtain the Laplace transforms of the partial derivatives 

du du du Fu 

x ot Ox or 
of the function u(x, 4), t = 0. Using the same procedure as that used to obtain the 
Laplace transform of standard derivatives in Section 5.3.1, we have the following: 
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ou er ° -st OU _d “ st 
(a) ax}=| « dar 2 « u(x, t) dt 


using Leibniz’s rule (see Modern Engineering Mathematics) for differentiation 
under an integral sign. Noting that 


co 


Lf u(x, t)} = U(x, s) -| e u(x, t) dt 


0 


we have 
ou d 
20 ne 2 
1 ee U(x, s) (9.29) 
(b) Writing y(x, t) = du/Ox, repeated application of the result (9.29) gives 


ol _d oe af 
a |= gen | . ve ) 


ox 
so that 
2 2 
see = eee rey, (9.30) 
ox dx” 


Ou | _ * ost Ou 
(c) aml a dt 


=[e “u(x, Do | e u(x, t) dt = [0 - u(x, 0)]+sU(x, s) 


0 


so that 


am = sU(x, s) - u(x, 0) (9.31) 


where we have assumed that u(x, f) is of exponential order. 


(d) Writing v(x, t) = du/ot, repeated application of (9.31) gives 
Aw = sV(x, s)- v(x, 0) 
ot 9 > 


= s[sU(x, 5) — u(x, 0)] — v(x, 0) 


so that 
ylfu =U 0 0 9.32 
Se f= PUG 8) ~ sux, 0) ~ Hs, 0) C2) 


where u(x, 0) denotes the value of du/ot at t = 0. 
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Example 9.16 


Solution 


Let us now return to consider the wave equation (9.4) 
20Uu = au 
a ery 
Ox ot 


subject to the boundary conditions u(x, 0) = f(x) and u(x, 0)/dt = g(x). Taking Laplace 
transforms on both sides of (9.4) and using the results (9.30) and (9.32) gives 
FEED = FUG 5) - 89 - sft (9.33) 
x 


The problem has thus been reduced to an ordinary differential equation in U(x, s) of a 
straightforward type. It can be solved for given conditions at the ends of the x range, 
and the solution can then be inverted to give u(x, ?). 

MAPLE or MATLAB can be used to assist with the transforms and the inverse 
transforms but considerable experience is needed to convert results to a simple form. 


Solve the wave equation (9.4) for a semi-infinite string by Laplace transforms, given that 
(a) u(x,0)=0 (x20) (string initially undisturbed); 

(b) u(x, OV/Ot=xe*" (x =0) (string given an initial velocity); 

(c) u(0O,t)=0 (t=O) (string held at x = 0); 

(d) u(x,)70 asx>oofort=0 (string held at infinity). 


Using conditions (a) and (b) and substituting for f(x) and g(x) in the result (9.33), the 
transformed equation in this case is 


c qd U(x, s) = 8° U(x, s)-x ee" 
dx 


By seeking a particular integral of the form 
U= Ox ewe 4 pew" 
we obtain a solution of the differential equation as 


—x/a 2 
U(x, s) =A er" 4 Be _ e [+ 2c la | 


cla -s° cla -s° 
where A and B are arbitrary constants. 

Transforming the given boundary conditions (c) and (d), we have U(0, s) = 0 and 
U(x, s) > 0 as x 4 ©, which can be used to determine A and B. From the second 
condition A = 0, and the first condition then gives 

2 
B = 2, ce 4 252 
(c/a - s°) 


so that the solution becomes 


2c la ete e : 2c’la | 


U(x, s)=— 
(cla? - sy (cla? - s’) (Cla - s’) 
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Fortunately in this case these transforms can be inverted from tables of Laplace 


transforms. 


Using the second shift theorem (5.45) together with the Laplace transform pairs 


@ 


2 2.2 
Ss -@ 


L{ sinh wt} = 


20 


ae cosh wt - sinh wt = 1 
( 2 


we obtain the solution as 


“=? [ - x) cosh (# - 
c a 


L{coshat} = — 


ery a 


-@) 


2 2 
Ss -@ 


2\2 


a 


+ tle +a) sinh @ -asinh (se 2) H(ct- ) 


where H(f) is the Heaviside step function defined in Section 5.5.1. 


9.3.4 Exercises 


Solve the wave equation 


subject to the initial conditions 
(a) u(x,0)=sinx (all x) 


ou 


(b) S!(,0)=0  Gallx) 


Use both the d’Alembert solution and the 
separation of variables method and show 
that they both give the same result. 


Find the separated solution of the wave equation 
Fu _1 du 
ax OF 

that satisfies the initial conditions 


u(x, 0) = 0, # (x, 0) = sin x(1 + cos x) 


Show that 


X+ct 


1-4ext-(x-e fy 14(x-ct)’ 1-(x+tcty 


www.2Ofile.org 


18 


19) 


20 


Hence deduce that the function satisfies the wave 
equation. Check that this differential equation is 
satisfied using MAPLE. 


The spherically symmetric version of the wave 
equation (9.4) takes the form 


1 du _du ,2du 
© of or ror 

Show, by putting v = ru, that it has a solution 
ru =f(ct—r) + g(ct +r) 


Interpret the terms as spherical waves. 


Using the trigonometric identity 
sin A cos B = } sin(4 — B) + } sin(A + B) 


rewrite the solution (9.28) to Example 9.14 as a 
progressive wave. 


Solve the wave equation 
Fu 1 Fu 
ax car 


subject to the initial conditions 


(a) u(x, 0) =0 (all x) 
ou x? 
(b) (x, 0) =xe (all x) 
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Find the solutions to the wave equation (9.4) 
subject to the boundary conditions 


(a) u(x, 0)/0t=0 for all x 


ine Se = 15 
(b) w=414x (-1<x<0) atr=0 
0 (|x| = 1) 


using d’Alembert’s method. Compare with 
Example 9.11. 
Compute the characteristics of the equation 


3Uy, + OU + Uy, = 0 


Show that the partial differential equation 


has solutions of the form u(x, t) = f(x + At) provided 
either A = 2 or A =-3. Given 


ou 


u(x,0)=x°-1 and (x, 0) = 2x for all x 


find the solution for w. 
The function u(r, ¢) satisfies the partial differential 


equation 


Ju ,2du_1ldu 


or ror 


where c is a positive constant. Show that this 
equation has a solution of the form 


r 
u= at) cos ot 
r 


where q is a constant and g satisfies 


2 2 
dg, 8 9 
dr Cc 


Show that, if u satisfies the conditions 
u(a, t) = B cos wt 
u(b, t)=0 


then the solution is 


_ Bacos at sin[ @(b - r)/c 
u(r, t= 


r sin[ @(b - a)/c] 


25 


26 


Za 


Use characteristics to compute the solution of the 
wave equation (9.4), with speed c = 1, given the 
initial conditions that for all x and t= 0 


(a) u=0 (b) du/ot = exp(-|x|) 


Use a time step of 0.5 to compute (on a spreadsheet 
or other package) the first four steps over the range 
—-3<x<3. 


Using the separated solution approach of 
Section 9.3.2, obtain a series solution of the 
wave equation 


Ju_1ldu 


ox ¢ or 
subject to the boundary conditions 
(a) u(0,t)=0 (¢> 0) 
(b) du(x, OVOt=0 (O<x<7) 
(c) u(t, =0 (t>0) 


(d) u(x, 0)=mx-x? (O<x <n) 


The end at x = 0 of an infinitely long string, initially 
at rest along the x axis, undergoes a periodic 
displacement asin wt, for t > 0, transverse to the 
x axis. The displacement u(x, ¢) of any point on 
the string at any time is given by the solution of 
the wave equation 

Qu _ 27u (x > 0,t> 0) 

ot ox 


subject to the boundary conditions 
(a) u(x,0)=0 (> 0) 

(b) du(x, 0VOt=0 (x >0) 

(c) u(0,t)=asin@t (t> 0) 
(d) Ju(x, |< ZL, ZL constant 


where the last condition specifies that the 
displacement is bounded. 

Using the Laplace transform method, show that 
the displacement is given by 


u(x, th=a sno - ‘J H(t - 2) 


where H(f) is the Heaviside step function. 
Plot a graph of u(x, ¢), and discuss. 
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Figure 9.19 Mesh 
points for a numerical 
solution of the wave 
equation. 
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Numerical solution 


For all but the simplest problems, we have to find a numerical solution. In Section 9.3.1 
we saw that characteristics give a possible numerical way of working by extending the 
solution away from the initial line. While this method is possible, it is difficult to program 
except for the simplest problems, where other methods would be preferred anyway. 
In particular, when characteristics are curved it becomes difficult to keep track of the 
solution front. However, calculus methods suffer because they cannot cope with dis- 
continuities, so that, should these occur, the methods described in this section will tend 
to ‘smear out’ the shocks. Characteristics provide one of the few methods that will trap 
the shocks when we use the fact that the latter are propagated along the characteristics. 

The numerical solution of ordinary differential equations was studied in some detail in 
Chapter 2. The basis of the methods was to construct approximations to differentials in 
terms of values of the required function at discrete points. The commonest approximation 
was the ‘chord approximation’ 


df(a) _ flath) -fla-h) 
dx 2h 


and for the second derivative 


d’f(a) ~ Math) -2f(ay+fla-h) 
dx* hr 


The justification of these approximations and the computation of the errors involved depend 
on the Taylor expansions of the functions. In partial differentiation the approximations are 
the same except that there is a partial derivative in both x and ¢ for the function u(x, f). 

Figure 9.19 illustrates a mesh of points, or nodes, with spacing Ax in the x direction 
and Aft in the ¢ direction. Each node is specified by a pair of integers (i, /), so that 
the coordinates of the nodal points take the form 


x,=at+iAdx, j=b+jAt 


and a and b specify the origin chosen. The mesh points or nodes lie on the intersection 
of the rows (j = constant) and columns (i = constant). 


Columns 
i-l i f+l 


th 


=V 
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The approximations are applied to a typical point P, with discretized coordinates 
(i,j), and with increments Ax = x,,, — x, and At= ¢,,, — 4, which are taken to be uniform 
through the mesh. We know that at the points A and B we can approximate 


(#) 2 WES =u = 1,3) 
A 


Ox Ax 
(#) — uitt/)-uli,/) 
Ox) Ax 


so that the second derivative at P has the numerical form 


Fu _ (duldx)y-(duldx), _ u(i+1,/) - 2u(i, /)+u(i- 1, /) 


ax Ax Ax? 
Similarly, 

Ju _ u(i, J +1) - 2u(i, /) tui, / - 1) 

or Ar 


Thus the wave equation @u/ot* = c’0’u/dx* becomes 


(dae I) S 2G, Dae Oj = I) 


At 
= op uit LJ) = 2uG, J) + uli - iL) 


2 


Ax 
which can be rearranged as 
uGy + l= 2uG, 7) — ua — 1) 
+ # [uit 1, j) — 2u(i, 7) + uli - 1, /)] (9.34) 
where 


A= cAtlAx 


Equation (9.34) is a finite-difference representation of the wave equation, and provided 
that w is known on rows j — | andj then w(i, 7 + 1) can be computed on row j + | from 
(9.34) and thus the solution continued. On the zeroth row the boundary conditions 
u(x, 0) = f(x) and du(x, 0)/dt = g(x) are known, so that f; = u(i, 0) and g; = Ou(i, 0)/0t 
are also known at each node on this row, and these are used to start the process off. 
From Figure 9.20, we see that 


du _ u(i, 1) - ui, -1) 
== 2 ? 9.35 
oa 2At 029) 
Figure 9.20 The first P 
rows of mesh points in us 
a numerical solution of | - 
the wave equation. Pe 
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Now (9.34) with j = 0 becomes 
u(i, 1) = 2u(i, 0) — (i, -1) + # [ui + 1, 0) — 2u(i, 0) + uli — 1, 0)] 
Since u(i, 0) =f, and u(i, -1) = u(i, 1) — 2Atg;, (9.34) now takes the form 
uli, y= (1 - RY + 32 (far thea) + At; (9.36) 


Thus the basic strategy is to compute row zero from u(i, 0) = f;, evaluate row one from 
(9.36), and then march forward for general row j by (9.34). 


Solve the wave equation 0u/dt? = c’Au/dx’ numerically with the conditions 
(a) u(x, 0)=sin(ax) (0<x <1) (initial displacement); 

(b) du(x, 0)J/Ot=0 (OSx <1) (initially at rest); 

(c) u(0O,)=ui,H=0 (t= 0) (the two ends held fixed). 

Use the values c = 1, Ax = 0.25, At= 0.1. 


Note that 7? = 0.16. The values at t= 0 are given by condition (a) 


x | 0 0.25 0.5 0.75 1 


u | 0 0.7071 I 0.7071 0 


The values at t= 0.1 (or j = 1) are computed from (9.36) with f= sin(zx) 
u(i, 1) = 0.84f; + 0.08( fir +f-1) 


and give 
x | 0 0.25 0.5 0.75 1 
u | 0 0.674 0.9531 0.674 0 


The first two rows are now complete, so formula (9.34) can be used for each of the 
subsequent times, for ¢ = 0.2 (or j = 2) 


u(i, 2) = 2u(i, 1) — u(i, 0) + 0.16[uii + 1, 1) — 2u(i, 1) + ui - 1, 1)] 
which gives 
x 0 0.25 0.5 0.75 1 


u 0 0.5777 0.8169 0.5777 0 


and for t = 0.3 (or j = 3) 


x | 0 0.25 0.5 0.75 1 
u | 0 0.4272 0.6042 0.4272 0 
and so on. 


This problem has an exact solution so the results can be compared with 
u(x, t) = sin(7x) cos(m2). 
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L_| 


Example 9.18 


Solution 


Figure 9.21 Solution 
of Example 9.18 with 
Ax = 0.2, 2 = 0.5 

for successive 

values of ct. 


Numerical calculations can be performed very efficiently with MATLAB: the ‘colon’ 
notation allows complex manipulations of sub-matrices to be done and makes the 
package ideally suited to this type of computation. The instructions, for n mesh 
points and general parameter L = 7, 


iol '5) Pit (0) 5 ALS % values in the example 
x=(Oei/ (i=l) eililp vessiaGeion)p SSeS wo Waslicaadl lime 
wve= (10), (tbat) 4 (22s IL |) seis (es (ib ein] sere ( [LS sim |) ) #2, OT 

Ssets up second line 
rerarh= 10) , aio vavs (| 28 rek— IL || ea ( LA Siav—aL || )) Eitye (Cage (AL Bia J) = 
sone | LA Siar || \yeeavs ( [Lk etal] }) j), (0) 

Ssets up the third line 
ASAE OWA = AVES a % prepares for subsequent 

lines 


produce the solution to this problem. Repeating the last two lines continues the solu- 
tion for ¢ incremented by At. 


Solve the wave equation @u/dt? = c’ Pu/dx’ for a semi-infinite string, given the initial 
conditions 


(a) u(x, 0)=xexp[-S(x- 1°] (x= 0) (string given an initial displacement); 
(b) du(x, 0)/Ot=0 (x =0) (string at rest initially); 
(c) u(0O,t)=0 (t=O) (string held at the point x = 0). 


Since g; = 0 in (9.36), only the one parameter J needs to be specified. Figure 9.21 shows 
the solution of wu over eight time steps with A = 0.5. It can be seen that the solution splits 
into two waves, one moving in the +x direction and the other in the —x direction. At a 
given time ¢ = 0.8/c, the uw values are presented in the table shown in Figure 9.22 for 
various values of A. We see that for 2 < 1 the solution is reasonably consistent, and 
we have errors of a few per cent. 
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Figure 9.22 

Table of values of u for 
a numerical solution 
of Example 9.18 with 
Ax = 0.2 and ct = 0.8. 
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x 0 0.2 0.4 0.6 0.8 1.0 

u(A = 0.25) 0 0.3451 0.4674 0.3368 0.1353 0.0236 
u(A = 0.5) 0 0.3487 0.4665 0.3318 0.1340 0.0272 
u(A = 1) 0 0.3652 0.4582 0.3105 0.1322 0.0408 
u(A = 2) 0 0.1078 0.3571 0.6334 0.5742 0.2749 


However, for A = 2 the solution looks very suspect. A further two time steps gives, at 
ct = 1.6, the solution 


x | 0 0.2 0.4 0.6 0.8 1 


uA=2) | 0 319 21.75 10.25 34.70 32.72 


Clearly the solution has gone wild! 


Looking back to Figure 9.20, we can attempt an explanation for the apparent diver- 
gence of the solution in Example 9.18. The characteristics through the points (x,_;, 0) 
and (x;,;, 0) are 


X,,=x- ct 
Xin, =X + ct 
which can be solved to give, at the point P, 
Xp = 5 ke +X;4) 
Clp = 7 Qin1 — Xi) = Ax 


Recalling the work done on characteristics, we should require the new point to be inside 
the domain of dependence defined by the interval (x,_,, x;,,). Hence we require 


tp = At 


so 
oe 24 (9.37) 


Indeed, a careful analysis, found in many specialist numerical analysis books, shows 
that this is precisely the condition for convergence of the method; it is commonly called 
the Courant, Friedricks and Levy (CFL) condition. 

The stringent condition on the time step At has always been considered to be a limita- 
tion on so-called explicit methods of the type described here, but such methods have 
the great merit of being very simple to program. As computers get faster, the very short 
time step is becoming less of a problem, and vector or array processors allow nodes to 
be dealt with simultaneously, thus making such methods even more competitive. 

There are, however, clear advantages in the stability of calculations if an implicit 
method is used. In Figure 9.19 the approximation to u,, may be formed by the average 
of the approximations from rows j + | andj — 1. Thus 


www.2Ofile.org 


766 PARTIAL DIFFERENTIAL EQUATIONS 


Example 9.19 


Solution 


[w(i, 7 + 1) — 2u(i, j) + ui, j — IVP A? 
=} [uit 1,j7+1)-2uG,j+ 1) +uG-1,7+ 1 
+ u(i+ 1, j—1)—2u(i, 7-1) + uGi- 1, 7 - 1)V/Ax? 


Assuming that uw is known on rows j andj — 1, we can rearrange the equation into the 
convenient form 

—Kulit 1,7 +1) +201 + Pui, 7 + 1) - Pui - 1,7 +1) 

= 4u(i, j) + Kui + 1,7 -1)- 20. + Dui, 7 - 1) + Kui - 1,7 - 1) (9.38) 
The right-hand side of (9.38) is known, since it depends only on rows j and j — 1. The 
unknowns on row / + | appear on the left-hand side. The equations must now be solved 
simultaneously using the Thomas algorithm for a tridiagonal matrix (described in 
Section 5.5.2 of Modern Engineering Mathematics). This algorithm is very rapid and 
requires little storage. It can be shown that the method will proceed satisfactorily for 
any A, so that the time step is unrestricted. The evaluation of rows 0 and | is the same 
as for the explicit method, so this can reduce the accuracy, and clearly the algorithm 
needs a finite x region to allow the matrix inversion. 


Solve the wave equation @u/dt? = c’Pu/dx* by an implicit method given 
(a) u(0,t)=0 (t=0) (fixed at x=0); 
(b) ul,t)=0 (t=0) (fixed at x= 1); 
(c) du(x,0)/0t=0 (0 x <1) (Zero initial velocity); 

1 =e 
(d) u(x, 0)= a 


0 otherwise 


(displaced at the one point x = +). 


Compare the solutions at a fixed time for various A. 


Here we have a wave equation solved for a string stretched between two points and 
displaced at a single point. 

The numerical solution shows the expected behaviour of a wave splitting into two 
waves, one moving in the —x direction and the other in the +x direction. The waves are 
reflected from the ends, and eventually give a complicated wave shape. 

The computations were performed with Ax = 0.125 and various A or At, with 
A= cAt/Ax. The values of u are given at the same time, T= Ax/c, for various A: 


x 0 0.125 0.25 0.375 0.5 0.625 0.75 0.875 1 
u(A = 0.2) 0 0.3394 0.2432 0.3412 0.0352 0.0019 0.0001 0 0 
u(A = 0.1) 0 0.3479 0.2297 0.3493 0.0344 0.0014 0 0 0 
u(A = 0.05) 0 0.3506 0.2254 0.3519 0.0341 0.0013 0 0 0 
u(A=0.025) | 0 0.3514 0.2243 0.3526 0.0340 0.0014 0 0 0 


Although the method converges for all A, the accuracy still requires a small A (or time 
step), but the value A= 0.05 certainly gives an accuracy of less than 1%. It may be noted 
that at the chosen value of T the wave has split but has not progressed far enough to be 
reflected from the end x = 1. 
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Equation (9.38) can be written in matrix form 

AU,,,=4U,;- AU, or U,,=4A'U)-U,, 
where the U vectors represent the whole row of wu values. This makes the problem 
ideal for MATLAB. The instructions, for n mesh points and general L = 7”, 


ea OR Ope 
é= [ky 2 (ilssby)) 


values for the example 
Oe a=2em—1, Ai, i-ileuti)= 
sets up matrix A 
loa |(i/2 I iby 2s Caaveta)y mer a=—2em-1, Ci, a-leusi)j=isp ceive! 
velo GC 1 O00 © © 0 O]’s weCru 


=p BSS (ial) 7 a; end 


sets up lines one and two 
B=inv (A) ;w=4*B*¥v-u 


WAVE WSN RE Sal 


valuates line thr 
continues the solution 


ae Of OF 


compute the solution. Repeating the last line continues the solution by an increment 


At. 


The methods described in this section all extend to higher dimensions, and some to 
nonlinear problems. The work involved is correspondingly greater of course. 


9.3.6 Exercises 


Use an explicit method to solve the wave 
equation @u/dt? = c’u/dx* for the boundary 
conditions 


(a) u(0, t) =0 
(b) ul, ) =0 


(t = 0) 
(t= 0) 
(c) ux,0)=0 (OSxS1) 


(d) ae 0) _ > 


l-x <x<1) 


Use Ax = At= i and study the behaviour for a 
variety of values of A for the first three time steps. 
Compare your result with the implicit version 

in (9.38). 


An oscillator is started at the end of a tube, 
and oscillations propagate according to the 
wave equation. The displacement u(x, f) 
satisfies 


Fu_1du 
ox Cc or 
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FE 


in0<x</, fort > 0, with the boundary conditions 


(a) u(0,)=asinot, ul,t)=0 (t>0) 


(b) u(x, 0) = 29) — 9 (0 <x<I) 


where c, a and @ are real positive constants. 
Show that the solution of the partial differential 
equation is 


ie h= asin ot sin[@(/- x)/c] 


sin(@l/c) 


si 2laco@ in( “= in( BEE) 
Dares GP -nrwe KT) \ I 
provided that w//mc is not an integer. 

Compare this solution with one computed using 
the explicit numerical method. Use a= 1, /= 1, 
c=1,@= it, Ax = 0.2 and At= 0.02 to evaluate 
u(x, 0.06). 


Solve the equation 


c Fu +2= Fu 
ax’ or 
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numerically, subject to the conditions Sil Solve the equation 
20u ou 
u=x(1—-x), dH <9 (0<x<1) att=0 EE] ¢ eel ae 
u=0 (x=0,1) fort>0 numerically, subject to the conditions 
Use u=x(1—x), 4 = for all x at t=0 
(a) an explicit method with Av = At = 0.2 and Use 
A=0.5; 
(b) an implicit method with Ax = Ar = 0.2 and (a) an explicit method with Ax = Ar = 0.2 and 
A=055. A= 0.5; 
(b) an implicit method with Ay = At = 0.2 and 
Compare your solution with that in Exercise 31. A=0.5. 


p Solution of the heat-conduction/diffusion equation 


In this section we consider methods for solving the heat-conduction/diffusion equation 
introduced in Section 9.2.2. 


9.4.1 Separation method 


It was with the aim of solving heat-conduction problems that Fourier (c. 1800) first used 
the idea of separation of variables and Fourier series. As indicated in Section 7.1, many 
mathematicians at the time argued about the validity of his approach, while he continued 
to solve many practical problems. 


In Section 9.2.2 we noted that the heat-conduction equation 
1 Ou 2 
->=V 9.5 
K Ot : >) 


has a steady-state solution U, provided there are no time varying inputs, satisfying 
V-U=0 

and appropriate boundary conditions. One useful way to write the general solution is 
u=U+v 


where v also satisfies (9.5) and the boundary conditions for uw — U. Certainly the heat- 
conduction interpretation supports this idea, and we base our strategy on first finding U 
and then determining the transient v that takes the solution from its initial to its final state. 
We note that v > 0 as t > ©, so that w — U, and an obvious method is to try an exponen- 
tial decay to zero. Thus, in the one-dimensional form of the heat-conduction equation 


Lav _ dv 
kK Ot ox 


we seek a separated solution of the special type discussed in Section 9.3.2, where the 
physics indicates a solution 


(9.39) 


v=e™ w(x) 
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Solution 
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which on substitution gives 

a dw 

Sw=o" 

K dx 

Letting @/«= 7, we can solve this simple-harmonic equation to give 


w=AsinAx+ BcosdAx 
and hence 
v=e“(AsinAx + Bcos Ax) (9.40) 


Taking the hint from Section 9.3.2, we expect in general to take sums of terms like (9.40) 
to satisfy all the boundary conditions. Thus we build up a solution 


v= > e “(AsinAx + BcosAx) (9.41) 


Solve the heat-conduction equation O7/dt = Ke T/dx* subject to the boundary conditions 
(a) Y=0atx=0 and for all t > 0 (held at zero temperature); 
(b) OT/dx =0 at x =/ and for all t > 0 (no heat loss from this end); 


(c) T=T7)sin(37nx/2/) at t= 0 and for 0 S x </ (given initial temperature profile). 


We first note that as tf > oo the solution will be JT = 0, so the steady-state solution is 
zero, and so from (9.40) we consider a solution of the form 


T=e“(AsinAx + Boos Ax) (9.42) 


In order to satisfy the boundary condition (a), it is clear that it is not possible to include 
the cosine term, so B = 0. To satisfy the condition (b) then requires 


OF _ 4 Roos he 0 (@w=l) 
Ox 


so that 
cosAl=0, or Al=(H+5)n (n=0, 1,2, 3,..+) 


leading to the solution 


T=Ae™ sn + on] 


We now compare the 7 from condition (c) with the solution just obtained at time ¢ = 0, 
giving 


205-22 . (3 
Ae” sa} + on] =T) sin( 32) 


The unknown parameters can now be identified as n = 1 and A = 7), and hence the final 
solution is 


T= Tyexp(- rt) sin( 32) 
4] 21 
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Example 9.21 


Solution 


Solve the heat-conduction equation ou/Ot = Ko’u/Ax? subject to the boundary conditions 
(a) u(0,t)=0 (t=O) (zero temperature at the end x = 0); 
(b) ul, =0 (t=0) (zero temperature at the end x = /); 


(c) u(x, 0)=u(4—x/l) (0<x</) (a given initial temperature profile). 


We are solving the problem of heat conduction in a bar that is held at zero temperature 
at its ends and with a given initial temperature profile. 

It is clear that the final solution as t > © is u = 0, so that U = 0 is the steady-state 
solution, and hence from (9.40) we seek a solution of the form 


u=e“(AsinAx + BcosAx) 


subject to the given boundary conditions. The first of these conditions (a) gives B = 0, 
while the second condition (b) gives 


sinAl=0, or Al=nn (n=1,2,...) 


Recalling that 1’ = a:/k, we find solutions of the form 
u=A em? sin( 2) (n _ 1, 2, i ) 


Clearly we cannot satisfy (c) from a single solution, so, as indicated in (9.41), we revert 
to the sum 


u=> A, ere” in (=) (9.43) 
n=1 


which is also a solution. 
Using the boundary condition (c), we then have 


1_X)_ 4 sin( 2™ 
(J | 5 4s sin( ae | 


and hence, by (7.33), 


1 
11A, = Up (: = 2) sin( dx = ° Rode) 
. l i ugl/nm (even n) 


Note again that we have used a periodic extension of the given function to obtain a 
Fourier sine series valid over the interval 0 < x < /. Outside the interval 0 < x </ we 
have no physical interest in the solution. Substituting back into (9.43) gives as a final 
solution 


Uo 1 4m? 12 1/1? 2 mn 
=— —_ —— 9.44 
u ‘ € sin( 7 ( ) 


m=1 


or, in an expanded form, 


Uo} —4«n2v/P . (2X loxP «(AMX —36Kn7 tl 
nate — sin( 20 )+4e ai sin( 40 +te mae 11 oe +... 
tT 


www.2Ofile.org 


Figure 9.23 Solution 
of Example 9.21 with 
T= t(4kr7/I’). 


Example 9.22 


Solution 
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uluty 


0.4 


0,2 


-0.2 


-0.4 


In Figure 9.23, u/up is plotted against x// at successive times T= t(4«17/1*) = 0, 0.5, 1, 1.5. 
Taking successive terms in the series for the values T= t(4K17/I*) = 0.5 and x/I = 0.2, 
we get 


| 1 term 2 terms 3 terms 4 terms 


u/Uy | 0.1836 0.1963 0.1956 0.1953 


and we see that three terms of this series would probably be sufficient to give three- 
figure accuracy. In all such problems some numerical experimentation is required to 
determine how many terms are required. For small ¢ and x we should expect to need a 
large number of terms, since the temperature at the end switches from ; Uy to 0 at time 
t= 0. It is well known, as we saw in Chapter 7, that discontinuities cause convergence 
difficulties for Fourier series. 


It may be noted in Example 9.21 that the initial discontinuity is smoothed out, as we 
expected from the physical ideas that we outlined in Section 9.2.2. 


Solve the heat-conduction equation du/ot = Ku/dx? in a bar subject to the boundary 
conditions 


(a) u(0,t)=0 (t 20) (the end x=0 is held at zero temperature); 
(b) ul,f=1 (t20) (the endx= 1 is at temperature 1); 


(c) u(x,0)=x(2-—x) (Ox <1) (the initial temperature profile is given). 


First it is clear that the final steady-state solution is U= x, since this satisfies (a) and (b) 
and also V’U = 0. Secondly putting uv = U + v, the new variable v satisfies (9.39), but 
now the boundary conditions on v are 
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9.4.2 


Example 9.23 


(a’) v(0, t) =0 
(b’) v0, ) =0 
(c’) u(x, 0) =x(2-—x)-x=x-X 


The appropriate solutions in (9.40) can now be selected; the condition (a’) gives B = 0, 
while the condition (b’), v(1, t) = 0, gives 


sinA=0, or A=nn (n=1,2,...) 
From (9.41) we then have 


= -Kn? nt. 
v= » a, e sin nux 


=I 


3 


and condition (c’) gives 
2 - : 
xx = Yan sin nx 
=1 


Determining the Fourier coefficient using (7.33), 


il 
la -| (x -x°) sin nx do = {nt (odd 7) 
2 n 


0 0 (even n) 


Thus the complete solution is 


u=x+8 y 1 gtr int sin n - 1) mx 
nm — (2n-1) 


n=1 


or, in expanded form, 


8 —-KT™ 


—25Kn2t 
u=x+—[e a 
Tl 


-9KTt 


sin3mx+-e sin5ax+...] 


Qe. as 1 
Sin TX xe Ds 


Laplace transform method 


As we saw for the wave equation in Section 9.3.3, Laplace transforms provide an altern- 
ative method of solution for the heat-conduction or diffusion equation. The method has 
the merit of dealing with the boundary conditions easily, but it suffers from the usual 
difficulty of performing the final inversion. The following example serves to illustrate 
these points. 


Using the Laplace transform method, solve the diffusion equation 


du_ Fu 
ax” 


(all x, t > 0) 
ot 


given that u(x, t) remains bounded and satisfies the boundary conditions 
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9.4 SOLUTION OF THE HEAT-CONDUCTION/DIFFUSION EQUATION 773 


(a) u(x,0)=0 (all x) 
(b) u(a, t) = TO(t) 


The problem models an infinite pipe, coincident with the x axis, that is initially 
filled with clean fluid. It is subjected to an instantaneous burst of contaminant 
injected at the point x = a > 0; the concentration of the contaminant, as it diffuses, is 
required. 

Using (9.30) and (9.31) and taking Laplace transforms gives 


2 
sU(x, 8) — u(x, 0) = cf UGS) 


xX 

which, on using condition (a), leads to the ordinary differential equation 
This is readily solved to give 

U(x, s) = Ae + Beem 
Since concentration remains bounded 

Ux, s)=Bev" for x > a and 

U(x, s)=Ae*™ forx<a 
Condition (b) then gives 

U(a, s) = L{T6(t)} = T= Bee 
so that 

B=Te®™ andsimilarly A=Te“"™ 
giving 

Ux, s)=Te’  forx > aand U(x, s)= Te  forx <a 


To find the solution u(x, f), we must invert the Laplace transform. However, in 
this case, the methods discussed in Chapter 5 do not suffice, and it is necessary 
to resort to the use of the complex inversion integral, which is dealt with in 
specialist texts on Laplace transforms (see also Chapter 8, Review exercise 7). 
Alternatively, we can turn to the extensive tables that exist of Laplace transform 
pairs, to find that 


¢! fe} _ i p32 eo bso 
\t 


We can then carry out the required inversion to give the solution 


ee = vad 132 gern ant (1 > 0) 
\ 
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It is possible to solve this example by using the extensive tables of Laplace trans- 
forms in MAPLE. It does, however, require some knowledge of the manipulative 
skills contained in the package to progress easily. 


It should be noted that the solution of Example 9.23 is not variable separable and could 
not be obtained from the methods of Section 9.4.1. 

To date all the problems studied have been restricted to heat flow in a rod. The ideas 
can be extended to spherical or cylindrical regions. Here one example will be considered 
that involves radially symmetric regions; see also Exercises 4 and 33. First the heat- 
conduction equation is written for a problem that only depends on the radial distance r 
from the origin and the time ¢. Now 


r=xrt+y+2 so dot = 9% 
ox 


To obtain the Laplacian 
Vf - of ae of 
ax oy ar 

when f= /(r, t) evaluate 


of _ for _xof 
ox orox ror 


and then the second derivative 


Bt. Osa) LaLa. d) 420d 


The y and z derivatives follow in a similar manner so the Laplacian becomes 


Vf =F mal +y +e) yetete te teat 


r r 


= ~_ 2 42(r2) 
ror or ror or 


The radially symmetric heat conduction equation for the temperature 7(r, f) is 


therefore 
| a(,20f) _ 1ar 
Pe a a Kot oo 


This radially symmetric form can be made to look very similar to the one-dimensional 
equation by writing 


pa Ar) 


r 


With this substitution 
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Solution 
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i2(-0 4 _ 120 
yor or) rKot 


and differentiating once more 
1/28 4.98 +r28) _ 190 
r\ or or OP) Kat 


Thus the equation for @(r, f) is just the usual one-dimensional heat-conduction 
equation 


JO _ 100 


ar (9.46) 


Methods developed earlier can now be used on this equation, but note that care must be 
taken at the origin where 7 = 0 since 


T= Or, t) 


r 


An example will illustrate some important points. 


Solve the radially symmetric heat conduction equation (9.45) 


(a) inthe region 0 <r < a subject to the boundary conditions T= 7, onr =a and T 
is finite in the region. The initial condition is 7(r, 0) = 0. 


(b) inthe region r > a subject to the boundary conditions T= 7, on r =a and T tends 
to zero as r tends to infinity. The initial condition is 7(r, 0) = 0. 


(a) To solve (9.45) put 
f=, 4 2S 
r 


then 0(r, f) satisfies (9.46) with the modified boundary conditions 0(a, t) = 0 and 
@(r, t)/r is finite at the origin. Clearly the only separated solution in (9.40) that 
can satisfy these conditions is 


@=e™ sindAr where a/k=¥ 
since sinAr/r— A as r > 0. The condition at r = a gives sin(Aa) = 0 so Aa = nn 
where x is a positive integer. Summing all solutions of this type gives 
1 — —Kn? 7? t/a? : 
T=T) toy, A,e sin(ntr/a) 
. n=1 
The coefficients A, are given by the initial condition which reduces to the Fourier 
series problem 
rT, = Y A, sin(ntr/a) 
n=1 


giving 
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(b) 


Aue 2aT,(-1)" 
T™ 
and finally 
2: T, —KT2t/la2 + —K4rt/a - 
T=T)- atte sin (ttr/a) - e tna" sin (2mr/a) +. . 
r 


As in part (a) we look to previous methods of solution. First note that it is only 
necessary to solve (9.46) with the modified boundary conditions O(a, t) = aT, and 
@(r, t) is finite for infinite r. The initial condition is O(r, 0) = 0. The problem is 
now very similar to Example 9.23 and Laplace transforms appears to be a sensible 
approach. The set up is precisely as in Example 9.23 (except for a change in the 
names of the parameters) to the point where 


sO(r, s) = 90,8) = 
dr 
where it should be noted that the zero initial condition has been used. The solu- 
tion is 
A(r, s) = Aewin +Be en 


which is now subject to the condition that @(r, s) is finite as r gets large implying 
that A = 0. To calculate B, the transformed condition at r =a gives O(a, s) = aT,/s 
and hence 


B- AT y isix)a 


Thus the solution for the transformed equation is 


= AT <shetns 
A(r, 8) = Fee Wwone-a 
AY 


The expression requires either access to extensive tables of Laplace transforms or 
to the tables in MATLAB or MAPLE. They give 


\e Ir- 
AG, y= aToerte Ga = 


so that 


Tr, ) = a erfe( 32 . 2) 
r 2 \Kt 


The error function 


2 -Vv 
erf(z)= —]e dv 
\t 
0 

and erfc(z) = 1 — erf(z) occur commonly in the solution of the heat-conduction 
equation (see also Section 7.6), and in statistics. It is well documented and appears 
as standard in MATLAB and MAPLE. The erfe function is illustrated in Figure 9.24 
and the solution 7/7, is plotted against r/a in Figure 9.25 for various times. 
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Figure 9.24 The 
function erfc(z). 


erfc(z) 


Figure 9.25 Plot 

of the solution to 
Example 9.24 for times 
A4xt/a’ = 0.01, 0.1, 1, 
10 and 100. 


Tit, 


rla 
9.4.3 Exercises 
32 Find the solution to the equation 33. The spherically symmetric form of the heat- 
conduction equation is 
du_ Fu q 
ot ax’ Uy, + Ey, =e u; 
r K 
satisfying the conditions : : 
By putting ru = v, show that v satisfies the standard 
(a) du/ox =0 atx =0 for all ¢ one-dimensional heat-conduction equation. What 
(b) uw=0 atx=1 forall t can we expect of a solution as r > co? 
(c) u=acos (mx) cos G tx) forO <x <1 Solve the equation in the annulusa =r <b 
when ¢ = 0 given that u(a, t) = Tp, u(b, t) = 0 for all t > 0 and 


www.2Ofile.org 


778 PARTIAL DIFFERENTIAL EQUATIONS 


34 


35 


36 


37 


38 


the initial condition u(r, 0) =0 fora <r < b. Show 


that the solution has the form 


7-3 Aue tt ‘sin(t =n 
b-a 


where A(b — a) = No. Evaluate the Fourier 
coefficients Ay. 


r= She aw 
r 


Show that u(x, f) = t“F(n), where n = x7/t is a 
solution of the partial differential equation 
au _ Fu 
Ot ax 
if F satisfies 
4nt i - aF =0 
diy dn 
Find non-zero values of o and « for which F = e*” 
is a solution. 


Show that u(x, 4) = f(x) sin(x — 
the heat-conduction equation 


dha 
ot ox 


provided that fand the constant 8 are chosen 
suitably. For a semi-infinite slab of uniform 
material occupying the region x = 0 construct 

the solution that satisfies (a) u is bounded as 

x — ee and (b) u(0, ft) = uy sin 2¢ (that is, a periodic 
temperature is imposed at x = 0). 


Bt) is a solution of 


Show that the equation 
0, = KO, — h(O— 8) 


can be reduced to the standard heat-conduction 
equation by writing wu = e"(@— @,). How do you 
interpret the term h(@— @,)? 

Use separation of variables to obtain a solution to 
the heat-conduction equation du/dt = Ko’u/dx’, 
given 


(a) du(0, j/Ox=0 (t= 0) 
(b) ui, =0 (t=0) 
(c) u(x, 0) = ug(4 -x/l) (0<x</l) 


Compare the solution with that obtained in 
Example 9.21. 


The voltage v at a time ¢ at a distance x along an 
electric cable of length Z with capacitance and 
resistance only, satisfies 


39 


40 


Fv _ldv 
ox’ Kat 


Verify that a form of the solution appropriate to the 
conditions that v = v) when x = 0, and v = 0 when 
x =L, for all values of f, is given by 


2 ae 
v= v(1 - 2) + Ye exp (=) sin( nn é) 
n=l L L 
where vy and the c, are constants. 


Show that if, in addition, v = 0 when t= 0 for 
0<x<L, 


29 
nt 


A uniform bar of length / has its ends maintained 
at a temperature of 0°C. Initially, the 
temperature at any point between the ends of 

the bar is 10 °C, and, after a time ¢, the temperature 
u(x, t) at a distance x from one end of the bar 
satisfies the one-dimensional heat-conduction 
equation 


du _1du 


(x > 0) 
ox’ Kat 


Write down boundary conditions for the bar and 
show that the solution corresponding to these 
conditions is 


u(x, t)= 


ay (1 - eos exp( = a £ =) 


n=1 
nx 
x sin( Bt ) 


The function (x, f) satisfies the equation 


20 - 28+ (H2H¢ 24120) 
ot ox” 


with the boundary conditions 
(a) O(-A, D) = o(h, 1) = 
(b) (x,0)=0 (-h<x<h) 


0 (t>0) 


where a, 6 and / are positive real constants. 
Show that the Laplace transform of the solution 


@(x, f) is 


ah. cosh[(s/a)!"?x 1} 
s 1/2 


cosh[(s/a) “h] 
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j+l 


> 
r-liitl x 


Figure 9.26 Mesh for 
marching forward the 
solution of the heat- 
conduction equation. 


Example 9.25 


Solution 
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Numerical solution 


As for the wave equation, except for the most straightforward problems, we must resort 
to numerical solutions of the heat-conduction equation. Even when analytical solutions 
are known, they are not always easy to evaluate because of convergence difficulties near 
to singularities. They are, of course, crucial in testing the accuracy and efficiency of 
numerical methods. 

We can write the heat-conduction equation 


ldu_ Fu 

Kot ox 
in the usual finite-difference form, using the notation of Figure 9.26. 
We assume that we know the solution up to time step 7 and we wish to calculate the 


solution at time step j + 1. In Section 9.3.5 we showed how to approximate the second 
derivative as 


(9.47) 


Ju _u(itlj -2u(i, j)+udi- 1,7 
ax Ax’ 
To obtain the time derivative, we use the approximation between rows j andj + 1: 
du _ ui, j+ 1) -uG,j 
Ot At 
Putting these into (9.47) gives 


u(i,j+1)-uGi, jf) _ uG-1,/)-2uG,j)+uG+1,/) 


2 


KAt Ax 
or, on rearranging, 
u(i, 7 + 1) = Auli - 1,7) + (1 — 2A)u(i, 7) + Auli + 1,7) (9.48) 


where A = KAt/Ax’. Equation (9.48) gives a finite-difference representation of (9.47), 
and provided that all the values are known on row /, we can then compute wu on row j + | 
from the simple explicit formula (9.48). 

First a simple example on a coarse grid. 


Use an explicit numerical method to solve the heat-conduction equation (9.47) subject 
to the boundary conditions 


(a) wu(0,t)=u(0,1)=0 (t= 0) (both ends held at zero temperature); 
(b) u(x, 0)=sin(tx) (0<x<1) (a given initial temperature distribution). 


Use the parameters At = 0.1, Ax = 0.25, k= 0.1. 


This problem has the exact solution u = a sin(1x), so the accuracy of the numerical 
solution can be checked easily. 
At t= 0 (or j = 0) the initial values come from the boundary condition (b) 
¥ | 0 0.25 0.5 0.75 1 
vu | 0 0.7071 1 0.7071 0 
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Example 9.26 


Solution 


Figure 9.27 Mesh 

for Example 9.26: 
u(7, 7) = 1 and 

u(0, 7) = u(1, /) for all 
j; Ax = 1/6.5 = 0.1538. 


Before proceeding further we first note that A= 0.16. At t= 0.1 (or j= 1) equation (9.48) 
becomes 


u(i, 1) = 0.16[u(i — 1, 0) + ui + 1, 0)] + 0.68u(, 0) 
which, on calculation for the values x = 0.25, 0.5, 0.75 or i= 1, 2, 3, gives 


x | 0 0.25 0.5 0.75 1 
u | 0 0.6408 0.9063 0.6048 0 


Note that at the ends, x = 0 and 1, the boundary condition (a) uw = 0 is imposed. 
At t= 0.2 (or j= 2) 
u(i, 2) = 0.16[u(i — 1, 1) + ui + 1, 1)] + 0.68u(i, 1) 

and performing the calculations 


x | 0 0.25 0.5 0.75 1 
u | 0 0.5808 0.8213 0.5808 0 


Similarly for ¢= 0.3 (or j = 3) 


x | 0 0.25 0.5 0.75 1 

u 0 0.5263 0.7444 0.5263 0 

u exact 0 0.5259 0.7437 0.5259 0 
and so on. 


In the last table the exact values have been included for comparison. 


Solve the heat-conduction equation (9.47) subject to the boundary conditions 
(a) du(0,t)/dx=0 (t=O) (no heat flow through the end x = 0); 
(b) uil,t)=1 (t=0) (unit temperature held at x = 1); 


(c) u(x,0)=x? (0<x<1) (a given initial temperature distribution). 


To fit the condition (a) most easily, we allow the first mesh space to straddle the ¢ axis 
as illustrated in Figure 9.27, where six intervals are used in the x direction. The mesh 
implies that Ax = 1/6.5 = 0.1538; condition (a) gives u(0, 7) = u(1, /) while condition 
(b) gives u(7, 7) = 1. 
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Numerical solution 
of Example 9.26 at 
time t = 20 Ax’/k, for 
two values of A. 
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Some simple MATLAB code produces the solution very quickly: 


Mea 7 bea) o249 values of the Example 
x= [0 2 5/ (aH il 5) sil / (ail. 5) el) pee “2 Sinitial data 
We (MO, a (i ( [Li sim 1] )) su ( [ Bhstat]] ) ) (Sain) Sw ( [2em=i]| )) , IL) § 
v(l)=v(2); 
Scomputes the first row 
Gave WO, ibe (Cem? || pu [Si sial] ) (Gla 2) ew ( [2 sim—1 |), ile 
v(l)=v(2); 
repeat this line of code to obtain successive rows 


The following table gives values for u at the three times t= 0, 0.2Ax’/k and 20Ax°/k, 
calculated with A = 0.2. These results are then compared at t = 20Ax’/k, computed with 
A taken to be 0.5. It may be noted that there are errors in the third significant figure 
between the two cases. 


j=9 0.0059 0.0059 0.0533 0.1479 0.2899 0.4793 —s «0.7160 
A=0.2 4 j=1 0.0154 0.0154 0.0627 0.1574 0.2994 0.4888 0.7255 
J=100 | 0.6817 0.6817 0.7002 0.7362 0.7874 =0.8510 = 0.9233 
A=0.5 j=40 0.6850 0.6850 0.7033 0.7389 0.7896 ~=—-0.8526 = 0.9241 


See ee 


For A= 0.54 we can obtain a solution that compares with the solution given, but for 
A = 0.6 the solution diverges wildly. Figure 9.28 shows a plot of the solution near the 
critical value of A at a fixed time ¢ = 20Ax7/x. As in the table, the solutions are accurate 
to about 1% for small A, but when A gets much above 0.5, oscillations creep in and the 
solution is meaningless. In Figure 9.29 a further graph illustrates the development of 
the solution in the two cases A = 0.2 and A = 0.55. One solution progresses smoothly 
as time advances, while the other produces oscillations that will eventually lead to 
divergence. 
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(a) (b) 
Figure 9.29 Plots of wu against x and ¢ from the solution of Example 9.26 with (a) A = 0.2; (b) A= 0.55. 


Comparing Example 9.26 with the numerical solution of the wave equation under- 
taken in Example 9.18, we observe similar behaviour for the explicit scheme, namely 
that the method will only converge for small enough time steps or 2. From (9.48) it 
may be noted that the middle term changes sign at A = 0.5, and above this value we 
might anticipate difficulties. Indeed, some straightforward numerical analysis shows 
that convergence is certain for A < 0.5. It is sufficient here to note that A must not be 
too large. 

To avoid the limitation on A, we can again look at an implicit formulation of the 
numerical equations. Returning to Figure 9.26, the idea is to approximate the x deriva- 
tive by an average of row j and row/ + 1: 


u(i, 7 + 1) — uli, 7) = AK — o)[ui - 1, 7) — 2uG, 7) + ui + 1,/)] 
+ a[udi-1,7+ 1) - 2uG,7+ )+udG+ 17+ DD} 


where 0 S a@ S | is an averaging parameter. The case ~ = 0 corresponds to the explicit 
formulation (9.48), while a= i is the best known implicit formulation, and constitutes 
the Crank—-Nicolson method. With =}, we have 


—Au(i -— 1,7 + 1) + 21 + Au, 7 + 1) — Aw + 1,7 + 1) 
= Au(i - 1,7) + 20. — Aju, j) + Aud + 1, /) (9.49) 


We know the solution on row /, so the right-hand side of (9.49) is known, and the 
unknowns on row (j + 1) have to be solved for simultaneously. 

Fortunately the system is tridiagonal, so the very rapid Thomas algorithm can be 
used. The method performs extremely well: it converges for all A, and is the best known 
approach to heat-conduction equations. 
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Repeat Example 9.25 but with an implicit scheme. 


At t= 0 (or = 0) the initial values come from the boundary condition and are identical 
for both the implicit and explicit formulations. 
At t=0.1 (or/j = 1) the three equations of the type (9.49) corresponding to x = 0.25, 
0.5, 0.75 ori= 1, 2, 3 are 
—0.16u(0, 1) + 2.32u(1, 1) — 0.16u(2, 1) = 0.16[u(0, 0) + u(2, 0)] + 1.68u(1, 0) 
—0.16u(1, 1) + 2.32u(2, 1) — 0.16u(3, 1) = 0.16[u(1, 0) + u(3, 0)] + 1.682(2, 0) 
—0.16u(2, 1) + 2.32u(3, 1) — 0.16u(4, 1) = 0.16[u(2, 0) + u(4, 0)] + 1.682(3, 0) 


After noting that the end boundary conditions give 


u(O, 0) = u(0, 1) = u(4, 0) = u(4, 1) = 0 


and the right-hand sides evaluated from the initial values, the equations can be written 
in matrix form as 


2.32 -0.16 oO |fuci,1)| | 1.348 
-0.16 2.32 -0.16|| w(2, 1)} =| 1.906 
0 -0.16 2.32}}u(3,1)| | 1.348 


The tridiagonal system can be solved to give 


x | 0 0.25 0.5 0.75 1 


u | 0 0.6438 0.9105 0.6438 0 


For the next time steps the matrix equation is identical, with the j-suffix advanced by 1 
at each time step and the right-hand sides re-evaluated from the most recently computed 
values of wu. Subsequent values are 


x | 0 0.25 0.5 0.75 1 
uatt=0.2 0 0.5862 0.829 0.5862 0 
uatt=0.3 0 0.5337 0.7547 0.5337 0 


and should be compared with the explicit solution in Example 9.25. 
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Example 9.28 


Solution 


EJ 


Figure 9.30 Solution 
of Example 9.28 with 
Ax =0.1, A= 0.3 using 
the Crank—Nicolson 
scheme. 


Solve the heat-conduction equation (9.47) using the implicit formulation (9.49) for the 
boundary conditions 


(a) u(0,t)=0 (t=O) (the end x= 0 is kept at zero temperature); 
(b) ul,t)=1 (t20) (the end x= 0 is kept at unit temperature); 


(c) u(x,0)=0 (Ox 1) (initially the bar has zero temperature). 


Here a bar is initially at zero temperature. At one end the temperature is raised to the 
value | and kept at that value. 


The matrix inversions are very tedious to perform, so again a package such as 
MATLAB solves the equations very quickly; note the use of the ‘colon’ notation. 


L=0.3; M=2*(1+L); N=2*(1-L); n=11; @values for the Example 

USZewos (i,m)? pe wita) a1; Sinitial data 

p=[-L M -L]; A=eye(n); for i1=2:n-1, A(i,i-1:1+1)=p; end 

gq=([L N LJ; B=eye(n); for 1=2:n-1, B(i,i-1:1+1)=q; end 
sets up the matrices in equation (9.49) 

DID) =sbioi (VN) Ep w= IDIDE wi solves for first row 

u=v; v=DD*u; %repeat this line of code for subsequent rows 


The results of the calculation are presented in Figure 9.30. At time step 0 the 
temperature distribution is discontinuous. The successive time steps 1, 10, 100 are 
shown, and the final distribution u = x is labelled oo. 
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9.4.5 Exercises 


41 Derive the usual explicit finite-difference 43 Given that u satisfies the equation 
e representation of the equation =] ue 
ul u 
du _ Ju a ax 
ot ax’ 


. . ; and is subject to the boundary conditions 
Using this scheme with At = 0.02 and Ax = 0.2, 


determine an approximate solution of the equation du =1 (=0,t>0) 
at t = 0.06, given that ox : 
u=x* when t=0 (0<x<1) u=0 (x=1,t>0) 
u=0 when x=0 (¢>0) u=x(1-x) (t=0,0<x<1) 
u=1 when x=1 (¢>0) derive a set of algebraic equations from the 
42 h explici ‘mplici ical implicit formulation in Section 9.4.4. Use the 
Vseibors cxplich and impel numerid implicit method by adapting the MATLAB 


segment in Example 9.28. Find the solution at 
t= 0.02 and 0.04 using the values Ax = 0.2 and 
At=0.02. 


formulations to obtain solutions of the heat- 
SI conduction equation subject to the boundary 
conditions 


(a) u0,)=0 (t2=0) (b) u1,n=e" (t=0) 
(c) ux,0)=0 (OSx<1) 


Compare the two results for ¢ = 1. 


Solution of the Laplace equation 


In this section we consider methods of solving the Laplace equation introduced in 
Section 9.2.3. 


9.5.1 Separated solutions 


It is much less obvious how to construct separated solutions for the Laplace equation, 
since there is less physical feel for the behaviour except that the solution will be 
smooth. We shall therefore work more formally, as in Section 9.3.2, and seek a solution 
of the Laplace equation 
ou , Zu 
ox” oy 
in the form 


u = X(x)Y(y) 


=0 (9.50) 


which gives on substitution 
2 2 
¥ dx ae xX ay =- 


0 
dx° dy 


or 
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Example 9.29 


Solution 


2 2 
GX eT peg (9.51) 
dx’ dy’ 


Now (d2X/dx’)/X is a function of x only and —-(d’Y/dy*)/Y is a function of y only. Since 
they must be equal for al/ x and y, both sides of (9.51) must be a constant, A. We 
therefore obtain two equations of simple-harmonic type 


2 2 
d¥oax, leary 


dx dy 

The type of solution depends on the sign of A, and we have a variety of possible solutions: 
A=-u <0: u=(Asinux + Bcospux)(Ce"” + De) (9.52a) 
A=ee>0: u=(Ae"+ Be™)(Csinuy+ Deospy) (9.52b) 
A=0: u = (Ax + B)(Cy + D) (9.52c) 


where A, B, C and D are arbitrary constants. Using the definitions of the hyperbolic 
functions, it is sometimes more convenient to express the solution (9.52a) as 


u=(Asinpx + Bcosux)(Ccoshuy + Dsinhuy) (9.52d) 
and (9.52b) as 
u=(Asinhux + Bcoshux)(Csinuy + Dcos Uy) (9.52e) 


The actual form of the solution depends on the problem in hand, as illustrated in the 
following examples. 


Use the separated solutions (9.52) of the Laplace equation to find the solution to (9.50) 
satisfying the boundary conditions 


u(x,0)=0 (O<x< 2) 
ux, 1)=0 (0<x< 2) 
u0,y)=0 (O<y< 1) 
u(2,y)=asin2my (O<y< 1) 


To satisfy the first two conditions, we need to choose the separated solutions that 
include the sin uy terms. Thus we take solution (9.52b) 


u= (Ae + Be™)(Csin uy + Dcos Uy) 
The first boundary condition gives 
(Ae“*+Be")D=0 (0<x <2) 
so that D = 0. Thus 
u=(A’e" + B’e™) sin Uy 
where A’ = AC and B’ = BC. The second boundary condition then gives 
(4 e"* + Be) sinu=0 (0<x< 2) 
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Solution 
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so that sin = 0, or u = nt with n as integer. Thus 
u=(A’e"™ + B’e"™) sinnty 
From the third boundary condition, 
(4+ B’)sinnty=0 (O<y< 1) 
so that B’ = —4’, giving 
u=A’(e’™ — e"™) sinnty 
= 24’ sinh nx sin nty 
The final boundary condition then gives 
24’ sinh 2nt sinnty=asin2my (0<y< 1) 


We must therefore choose n = 2, and a = 24’ sinh 2nn = 24’ sinh 4n, or 2A’ = a/sinh 4n. 
The solution is therefore 


sinh 21x 


u=asin2ty 
» sinh 41 


Solve the Laplace equation (9.50) for steady heat conduction in the semi-infinite region 
0 <y <1,x = 0 and subject to the boundary conditions 

(a) u(x,0)=0 (x20) 

(b) u(x,1)=0 (x20) / (temperature kept at zero on two sides and at infinity); 
(c) u(x,y) 790 asx 

(d) u(0,vy)=1 (0<yS<1) (unit temperature on the fourth side). 


Clearly from condition (c) we need a solution that is exponential in x, so we take 
(9.52b): 


u=(Ae"*+ Be™)(Csinuy + Dceos iy) 
and since the solution must tend to zero as x > -, we have A = 0, giving 
u=e"(C’ sin uy + D’ cos Uy) 


where C’ = BC and D’ = BD. Condition (a) then gives D’ = 0, and (b) gives sin = 0, 
or U=nt (n= 1, 2,...), so the solution becomes u = C’e”™ sinnay (n= 1, 2,...). 
Because of the linearity of the Laplace equation, we sum over n to obtain the more 
general solution 


u= > C’e"™ sinnty 


n=1 


Condition (d) then gives, as before, a classic Fourier series problem 


i=¥) C’sinnty (0<yS< 1) 
n=1 
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Figure 9.31 Solution u 
of the Laplace equation 
in Example 9.30. 


so that, using (7.33), 


1 
c.=2| sinmy ay = | (odd 7) 


r 0 (even n) 


The complete solution is therefore 


u= py = @ PY" sin(2n — 1)ty 


n=1 
or, in expanded form, 


= Tt T. 


u= =(6 “sin ty + te? * sin 3my + ie “sin Smy+...) 

T 
In Figure 9.31 the solution u(x, f) is plotted in the (x, y) plane. Because of the discon- 
tinuity at x = 0, for x = 0.05 thirty terms of the series were required to compute wu to 


four-figure accuracy, while for x = 1, one or two terms were quite sufficient. 


It is clear from Example 9.30 that the solutions (9.52) can only be used for rectangular 
regions. For various cylindrical and spherically symmetric regions separated solutions can 
be constructed, but they need more complicated Bessel and Legendre functions. The solu- 
tions have the same structure as for rectangular regions and follow the general theory 
of orthogonal functions discussed in Section 7.7.2. For instance the study of Legendre 
polynomials is required for problems similar to Example 9.24 when angular dependence 
is included. There is great merit in calculating exact solutions where we can, since they 
give significant insight. However, with modern computing techniques it is certainly not 
necessarily quicker than a straight numerical solution. 


Example 9.31 — Solve the Laplace equation (9.50) in the region 0 < x < 1, 0 <y < 2 with the conditions 
(a) ux, 0)=x (OSxS1) 
(b) u(x,2)=0 (OSx<1) 
(c) u0,y)=0 Oy 2) 
(d) Ou(l,y)/ox=0 (Oy S 2) 
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Solution 


Figure 9.32 

Region and boundary 
conditions for 
Example 9.31. 


The steady heat-conduction interpretation of this problem, looking at Figure 9.32, gives a 
zero temperature on ABC, an insulated boundary on CD and a linear temperature on AD. 

Of the solutions (9.52), we require zeros on AB and zero derivative on CD, so we 
might expect to use trigonometric solutions in the x direction and exponential (or 
equivalently sinh and cosh) solutions in the y direction. We therefore take a solution of 
the form (9.52d): 


u=(Asinux + Bcos ux)(Ccoshuy + Dsinh uy) 
From condition (c), we must take B = 0, giving 

u=(C’ coshuy + D’ sinh Wy) sin ux 
where C’ = AC and D’ = AD. Condition (d) then gives cos = 0 or 

M=(n+3)0 (4 =0;.1,.25 324.) 
so the solution becomes 

u =[C’ cosh(n + ; \ty + D’ sinh(n +} ay] sina + })ax (n=0,1,2,...) (9.53) 
To satisfy condition (b), it is best to rewrite (9.53) in the equivalent form 

u = sin[(n + + mx] {E cosh[(n + } )n(2 — y)] 

+ Fsinh{(n + })m(2—y)]}} (n=0,1,2,...) 


We see that (b) now implies E = 0, so that our basic solution, summed over all n, is 


= 3 F, sin[(n + })nx] sinh[(n + 3)n(2 - y)] 
n=0 


The final condition (a) then gives the standard Fourier series problem 


x= Vi sinh[(2n + 1) 1] sin[(n +!) nx] 
n=0 


so that, using (7.33), 
1 


iF, sinh(2n+ 1)m = | x sin[(n + +)nx]dx = : ; 
0 : T (n+) 


; 1 
sin(n + 5)1 


The solution in expanded form is therefore 
8 _, Sinhjn(2-y) — sin $rx sinh $x (2 — y) 


u=—- 5 TX 
a sinh 1 9 sinh 31 


+ sin 57x 


sinh $7 (2 - y) 7 
25 sinh 51 a 


Curiously, Laplace transform solutions are not natural for the Laplace equation, since 
there is no obvious semi-infinite parameter. Even in cases like Example 9.30, where we 
have a semi-infinite region, the Laplace transform in x requires information that is not 
available, see Section 9.8.2. 
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Example 9.32 


Solution 


Figure 9.33 
Complex-variable 
solution to the 
Laplace equation 
in Example 9.32, 
showing the 
streamlines of 
flow into a corner. 


Another technique for solution of the Laplace equation involves complex variables and 
was discussed in Section 4.3.2. It is a method that was very widely used in aerodynamics 
and in electrostatic problems. Since the advent of modern computers, with highly efficient 
Laplace solvers, the method has fallen somewhat into disuse. It was the cornerstone of 
all early flight calculations and, for those interested either in the historical context or in 
the beautiful mathematical theory, the study of complex-variable solutions is essential. 
The real and imaginary parts of a differentiable function f(z) of the complex variable 
z=x+Jy automatically satisfy the Laplace equation. Example 9.6 showed a solution 
that was interpreted as the flow past a cylinder; the function was obtained from the 
imaginary part of 


roared 


It is then possible to use the Kutta—Joukowski transformation to transform the circle to 
an aerofoil shape and the lift and drag on the aerofoil can be computed. Example 9.32 
illustrates a much simpler situation. 


If f(z) = (x, y) + jw(x, y) is a complex function of the complex variable z = x + jy, 
verify that @ and y satisfy the Laplace equation for the case f(z) = z’. Sketch the 
contours of @= constant and y= constant. 


Now 
f@)= 2? = (x? —y’) + j2xy 

and thus 
gax’-y’, waxy 

It is trivial to differentiate these functions, and both clearly satisfy the Laplace equation: 
V’o=0, VWy=0 

The contours of @ and yw are plotted in Figure 9.33. They are both hyperbolas, which 


intersect at right angles. The usual interpretation of these solutions is as irrotational 
inviscid fluid flow into a corner. 


y yw = constant @ = constant 


www.2Ofile.org 


Example 9.33 
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If we now try to solve the Poisson equation (9.10), which is an extension of the 
Laplace equation with heat sources/sinks f(x, vy) on the right-hand side 


ou ou 
oe +l =f(x,y 
ot ay? S(x,y) 


the problem becomes harder as illustrated in Example 9.33. 


Solve the Poisson equation 


du ,du «1X Ty 


— +— =-sin~cos 
ax’ ay’ a b 


in the rectangle 0 < x <a, 0 <y S b given the boundary conditions 


u=Oonx=0, u=f(y)onx=a 


ou =0 ony=Oandy=b 
oy 


Physically the problem can be interpreted as a heated plate with the temperature 
specified on the two boundaries x = 0 and x = a and with insulated boundaries y = 0 
and y = b. 

The general strategy is to find a ‘particular integral’ to eliminate the term on the 
right-hand side, compute the new boundary conditions and then solve the residual 
Laplace equation. In the present case choose 


U= Ksin& cos ™ 
a b 


Substitute into the Poisson equation to give 


2 2 
VU= -K(E +2 ) sin cos"? 
a 


py oat 
and hence 
=f of 1 1 
KvV= T & +2) 
Now put 
u=U+v 


so that v satisfies the Laplace equation 
Vv=0 


and the boundary conditions remain the same 


v=Oonx=0, v=f(y)onx=a 


dv =0 ony=Oandy=b 
oy 


www.2Ofile.org 


792 PARTIAL DIFFERENTIAL EQUATIONS 


We are now back to a standard Laplace equation problem that can be solved by separation 
of variables. From the solutions (9.52) choose 


1 
=-A 
v 5 or 


nTMX on 
v = A, sinh — cos —= nny n=1,2,3 
a 


These solutions satisfy three of the boundary conditions, just leaving 


v=f(y) onx=a 


to be satisfied. The usual infinite sum of terms is constructed 


soil yA. sinh cog 
v= Ax + >) 4, sinh oor 


n=1 


so that on x = a the remaining boundary condition gives the usual Fourier cosine series 
problem 


af > iA. 6i nmy 
SY) = 5408 + > (A, sinhnt) cos 


n=1 


In Chapter 7 (7.30) and (7.31) give 


b 


ay = Aya = © | syd 


b 
and a, = A, sinh nn = 3] rycos 2!) oe ae 
0 


The final solution is given by 


sin™* cos rire co i 
a + ATX n 
x +> ——— sinh oo 


“Aa gl iy *e Dinan a b 
a 


The method of solution described in Example 9.33 can be quite difficult and clumsy. 
Finding the ‘particular integral’, U, is not always easy even for simple right-hand sides. 
If U can be found, the new boundary conditions on v can become very awkward, 
often further substitutions need to be made to bring the problem to tractable form. In 
Example 9.33 the right-hand side was carefully chosen to avoid this extra difficulty. An 
alternative method using Green’s functions will be considered in Section 9.7.2. The 
solution turns out to be very neat with the right-hand side and the boundary conditions 
appearing naturally in various integrals. However, although neat, the computation of 
the Green’s function is just as difficult as the method described in Example 9.33. 


www.2Ofile.org 


44 


45 


46 


47 


9.5.2 Exercises 


Use the separated solutions (9.52) to solve the 
Laplace equation 


du, du 


ax ay 


in the region 0 < x < 1, y = 0 given the boundary 
conditions 


(a) w=Oonx=Oandx=1 
(b) u>0asye 
(c) w=sin*(tx) on y =0 


(y 2 0) 
(0<x<1) 
(0<x<1) 
(Note: the identity sin’@ = 4 (sin5@— 5 sin30+ 
10 sin @).) 


Show that the function u(x, y) = e™”[y cos(my/2) 
—xsin(my/2)] satisfies the Laplace equation 


Ju, dug 
Ox oy 
and the boundary conditions 
(a) w=Oony=0 (x = 0) 
(b) uw=-xe™’ ony=1 (x = 0) 


(c) w=ycos(my/2) onx=0 (0<y<1) 
(d) u>0asx 


Show that the function @(x, y) = xy satisfies the 
Poisson equation 


Fo , Fb yy 
ax’ oy 


By putting @ = u+x’y, show that u satisfies the 
Laplace equation. Find the solution for @ in the unit 
square which satisfies the boundary conditions 


(x, 0) =0 


tes \ toro x1 
o(x, 1) = x + sin tx 


(0, y) = 0 


| toss 
oC, y) = y 


Show that 
u(r, 8) = Br" sinn@ 


satisfies the Laplace equation in polar coordinates, 


1 1 
u,, + —U, + [Ug = 0 
r r 
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48 


49 


50 


Syl 


Determine wu that is both finite for 7 S a and periodic 
in 0, given that 


u(a, @) = sin’@ = 3 sin 6 — } sin30 
Verify that 


x+y-1 
v= 


a ae 
x+y +2x41 


¢ + y +2x4+1 
both satisfy the Laplace equation, and sketch the 
curves u = constant and v = constant. Show that 


ipo 


z+1 


where z = x + jy. 


(Hadamard example) Show that the Laplace 
equation 


Fuldx* + Puldy? =0 


with u(0, v) = 0, (0, y) = (1/n) sinny (n > 0) has 
the solution 


u(x, y) = 4 sinh nx sin ny 
n 
Compare this solution, for large n, with the solution 
to the ‘neighbouring’ problem, when u(0, y) = 0, 
u,(0, vy) = 0, and the solution u(x, y) = 0. 


Solve the Laplace equation @u/dx* + Pu/dy? = 0 
in the region 0 <x < 1,0 <y< 1 subject 

to the boundary conditions u(0, vy) = 0, u(x, 0) = 0, 
u(1, vy) = 1, u(x, 1) = 1 by separation methods. 


A long bar of square cross-section 0 =< x <a, 

0 <y <a has the faces x =0,x =a andy =0 
maintained at zero temperature, and the face 

y =a ata control temperature wu). Under steady-state 
conditions the temperature u(x, y) at a point ina 
cross-section satisfies the Laplace equation 


2. 
ox oy 
Write down the boundary conditions for u(x, y), and 
hence show that u(x, y) is given by 


=0 


4u cosech(2n + 1) 
ae) aero 


n=0 


x sin (2m - ya sin (Qn sf ve 
a a 
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52 


53 


Heat is flowing steadily in a metal plate whose 
shape is an infinite rectangle occupying the 

region —-a <x <a, y > 0 of the (x, y) plane. The 
temperature at the point (x, y) is denoted by u(x, y). 
The sides x = +a are insulated, the temperature 
approaches zero as y — oo, while the side 

y = 0 is maintained at a fixed temperature —T for 
-—a <x <0 and T for 0 < x <a. It is known that 
u(x, y) satisfies the Laplace equation 


dis, dhe 
ax oy 


and the boundary conditions 


=0 


(a) u>0asy>o forallxin-a<x<a 
(b) du/dx = 0 when x = +a 
Ff tae 2) 


(c) u(x, 0) -| 
T (0<x<a) 


Using the method of separation, obtain the solution 
u(x, y) in the form 


_4T< 1 1,1 
u(x, y) = os Y ee e190 + ye 
n=0 
x slo + ye 
a 


A thin semicircular plate of radius a has its bounding 
diameter kept at zero temperature and its curved 
boundary at a constant temperature 7). The steady- 
state temperature 7(r, 0) at a point having polar 
coordinates (r, 0), referred to the centre of the 
circle as origin, is given by the Laplace equation 


OT 1o0T 1¢0T 
—+->+--5=0 
or ror rae 


9.5.3. Numerical solution 


54 


Assuming a separated solution of the form 
T= R(r)O(8) 
show that 


2n+1 


y= toe lay 5 
T(r, 0) = = py ore sin(2n + 1)0 


The Laplace equation in spherical polar coordinates 
(r, 8, @) takes the form 


If Vis only a function ofr and 0, and V takes the form 


V=R(r)y(x), where x=cos@ 
show that 

d(2dR)\_ 

rail an) =k(k+1)R 


2 
(1-2) 2x + e+ Dy =O 
LX, 


where & is a constant. 

The function V satisfies the Laplace equation in 
the regiona Sr <b. Onr=a, V=0 and on 
r=b, V=asin’@, where ois a constant. Given that 
solutions for y are 


1 (k=0) 
yousx (k=1) 
4(3x°-1) (k=2) 


find V throughout the region. 


Of the three classical partial differential equations, the Laplace equation proves to be 
the most difficult to solve. The other two have a natural time variable in them, and it is 
possible, with a little care, to march forward either by a simple explicit method or by an 
implicit procedure. In the case of the Laplace equation, information is given around the 
whole of the boundary of the solution region, so the field variables at a// mesh points 
must be solved simultaneously. This in turn leads to a solution by matrix inversion. 
The usual numerical approximation for the partial derivatives, discussed in Section 9.3.5, 
are employed, so that the equation 


du, Pu 


ax — 
vy 


(9.54) 
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Figure 9.34 Five- 
point computational 
module for the Laplace 
equation. 


Example 9.34 


Solution 
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at a typical point, illustrated in Figure 9.34, becomes 
u(i + 1, j/) — 2uGi, j/) +uGi- Lj 4 i,j +1) — 2u(i, 7) + ui, j- 1 
Ax’ Ay’ 
For the case Ax = Ay rearranging gives 
4u(i, jJ)=udi+ 1,fj)tudi-1yj)+uG,j+D+uGj7-) (9.55) 


In the typical five-point module (9.55) the increments Ax and Ay are taken to be the 
same and it is noted that the middle value w(i, 7) is the average of its four neighbours. 
This corresponds to the absence of ‘hot spots’. We now examine how (9.55) can be 
implemented. 


Solve the Laplace equation (9.54) in the square region0 <x < 1,0 <y < 1 with the 
boundary conditions 


(a) vw=0onx=0 (b) uw=lonx=1 


(c) uw=0 ony=0 (d) uw=Oony=1 


For a first solution we take the simplest mesh, illustrated in Figure 9.35(a), which 
contains only four interior points labelled u,, u5, u; and us. The four equations obtained 
from (9.55) are 


4u,=04+0+uy+Uy 
4u,=0+1+u,+u, 
4u,=1+0+u,+ uy 
4u,=0+0+u, +4, 
which in turn can be written in matrix form as 
4 -1 0O -Il) |u, 
-1 4 -1 9 Of fu} _ 


GO - 4A cet) le, 
-—1 0 =1 Ally, 
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O 0.333 0.667 1.000 * 


(a) 


Figure 9.35 Meshes for the solution of the Laplace equation in Example 9.34: (a) a simple mesh containing 4 interior 
points; (b) a larger mesh with 49 interior points. 


This has the solution uv, = 0.125, u, = 0.375, u; = 0.375, uy = 0.125. A larger mesh 
obtained by dividing the sides up into eight equal parts is indicated in Figure 9.35(b). 
The equations now take the form 


4u(1, 1) =u(2, 1)+ 0 + u(1, 2)+0 


4u(2, 1) = u(3, 1) + ud, 1) + uQ2, 2)+ 0 


4u(7, 1) =1 + u(6, 1) + u(7, 2) +0 


4u(1, 2) = u(2, 2) + 0 + u(1, 3) + ud, 1) 


Au(2, 2) = u(3, 2) + u(1, 2) + u(2, 3) + u(2, 1) 


4u(7, 2) = 1 + u(6, 2) + u(7, 3) + u(7, 1) 


We thus generate 49 linear equations in 49 unknowns, which can be solved by any 
convenient matrix inverter. The matrices take the block form 


‘4 <¥ ¢§ OS & BH 
414 4 0 @ 6 oO 
0-1 4-1 0 0 0 
A=|0 0-1 4 -1 0 0 
0 0 0-1 4 -1 #0 
0 0 OG = & a 
0 0 0 0 0-1 4 
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Figure 9.36 

The solution of 
Example 9.34. 

The solution is 
symmetric about the 
line j = 4; the solution 
with Ax = 0.125 is 
given in the upper half 
and the solution with 
Ax = 0.25 is shown 
in parentheses in the 
lower half. 


cooclcmcUchCcmhYE CUO 
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so that the equations become 


A 


ocococUccolUCOUWWM 


cco eco USB DW 


cco WP Wo 


0 0 0 
0 0 0 
-1 0 0 
a ae 
0 0 -l 
0 0 0 
0 0 0 
| (1, k) | 
u(2, k) 
U, = 
| u(7, k) | 
0 & » 0) 
0 0 0 0 
B 0 0 0 
AB 00 
BA BO 
0 BAB 
00 BA 


rPeooooo:s 


BAA nh Aas, 


(9.56) 


The matrix equation (9.56) can be solved by an elimination technique or an iterative method 
like successive over-relaxation (SOR). As indicated in Section 5.5.4 of Modern Engineer- 
ing Mathematics, SOR is the simplest to program, and elimination techniques are best 
performed by a package from a computer library. For the current problem we present the 
solution in Figure 9.36, where the cases Ax = : and Av = Fi are both shown. It may be seen 
from this example that the accuracy of the solution is quite tolerable when the cases Ax = ; 


j values 

8 | 0 O 0 0 0 0 0 0 1 

7 | 0 0.017 0.038 0.064 0.103 0.164 0.269 0.4831 

6 | 0 0.032 0.069 0.117 0.184 0.282 0.431 0.661 1 

5 | 0 0.042 0.089 0.150 0.233 0.350 0.512 0.731 1 

4/0 0.045 0.096 0.162 0.250 0.371 0.536 0.749 = 1 

(0.098) (0.250) (0.527) 

3 

2 | 0 (0.071) (0.188) (0.429) 1 

1 

0 | 0 0 0 0 1 
0 1 2 3 4 5 6 vi 8 
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EJ 


x 


i=-li=0 i=l i=2 
Figure 9.37 
Fictitious nodes, i=—1, 
introduced outside 

the boundary. 


and Ax = i are compared. Note the averaging behaviour of the Laplace equation and 
observe that the discontinuity in the corner does not spread into the solution. The corner 
nodes are never used in the numerical calculation, so the discontinuity is avoided. 


The solution of the Laplace equation is often required so packages like MATLAB have 
the machinery to set up the solution for simple regions. For the 9 x 9 problem, with 
49 unknowns, the code is listed; note that the MATLAB numbering of the nodes is 
different from the text. 


G=numgrid(‘S’,9) % sets up the numbering for a 9 x 9 square 
A=delsq(G); % stored as a sparse matrix 

tlasi=yqsrcoys (AS), 1) piclas(4A seus), ljaomas (7, je 3 cenjemires! rls 
A\rhs % gives the quoted solution 


Because of its simplicity, SOR is an attractive method for solving Laplace-type prob- 
lems. Equations (9.55) are rewritten with an iteration superscript as 


u"'lG,j)=u"i, j) + Fw" +1 jtu'G-Lyjt+u'Gj+l 
+u"(i, 7 — 1) - 4u"G,7)] (9.57) 


and w is a relaxation factor, discussed in Chapter 5 of Modern Engineering Mathematics. 

Knowing all the u(i, 7) at iteration n, we can use (9.57) to evaluate u(i, 7) at iteration 
n+ 1. Normally the u(i, 7) are over-written in the computer as they are computed, so that 
some of the ms in the right-hand side of (9.57) become (n + 1)s. The order of evaluation 
of the is and js in (9.57) is critical, but the most obvious methods by rows or columns 
prove to be satisfactory. 

A great deal is known about the optimum relaxation factor w. It is closely related to 
the value of the maximum eigenvalue of the matrix associated with the problem. For 
square regions with unit side and with wu given on the boundary and equal mesh spacing 
it can be shown that w = 2/(1 + sinAx) is the best value. For other problems this is 
usually used as a starting guess, but numerical experimentation is required to determine 
an optimum or near-optimum value. 

We have only considered u to be given on the boundary, and it is essential to know 
how to deal with derivative boundary conditions, since these are very common. Let us 
consider a typical example: 


Ou _ _ 
ee =g(y) onx=0 


We then insert a fictitious line of nodes, as shown in Figure 9.37. Approximately, the 
boundary condition gives 


u(1, 7) — u(-1, 7) = g(y,)2Ax 
so that 
u(-1, j) = u(1, 7) — 2Ax g(y;) (9.58) 


Equations (9.55) or (9.57) are now solved for i = 0 as well as i > 0, but at the end of a 
sweep u(—1, 7) will be updated via (9.58). 
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Example 9.35 


Figure 9.38 The mesh 
for Example 9.35. 


Solution 
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Solve the Laplace equation (9.54) for steady-state heat conduction in the unit square, 
given that 


(a) du/dx=+-y, x=0 (steady heat supply on this boundary); 
(b) u 
Use Ax = Ay =}. 


0onx=1,y=0,y=1 (zero temperature on the other three sides). 


Labelling the six unknown values w,, 5, . . . , Ws aS Shown in Figure 9.38, equation (9.55) 
gives 


4u,=O0+u,+u,+u_y 

4u,=O0+u,+us+uU, 

4u,=04+0+u,+ uw 

4u, =u, tus +0+U_s 

4us =U, +ug t+ 0+ Uy 

4u,=u,+0+0+4+u;5 
The values u_, and u_; are evaluated from boundary condition (a) 

Us—us=2h(h—2) =-i, w—u,=2h(1-1) =} 
so the equations become 

4u, = 2u, + uy - : 

4u, = uy, + uy + Us 

4u; = Uy + Us 

4u, =u, + 2us5+ 7 

4us = Uy + Uy + Ue 

4u,g = uy + Us 


Thus there are six linear equations in six unknowns, which can be solved by any con- 
venient method. For instance, SOR as suggested in (9.57) gives the set of equations 
with iteration counter n and relaxation factor w 
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Example 9.36 


Solution 


un! =u, t+ 2(2u3 + uy - 7 — 4u') 
uy = uy + Fur + us + us — 40) 
Wi uh + BU 4 ue — au) 

uy =u + S(uy + Qu +4 - 4ui) 
ul a uh + (us + ul! + ul — au’) 


n+l n yy ntl 
Us = Ue + F(Uy 


fp un — 4uh) 

The equations are diagonally dominant so the iterations converge quickly; six significant 
figures are obtained in 11 iterations with w = 1 and at near optimum w = 1.2 in 8 
iterations. 


u; Uy U3 U4 Us Us 


t —0.02424  —0.005 05 —0.001 01 0.024 24 0.005 05 0.001 01 
h=} —0.031 21 —0.005 17 —0.000 77 0.031 21 0.005 17 0.000 77 
h=3 —0.03357  —0.005 22 —0.000 68 0.033 57 0.005 22 0.000 68 


The expected symmetry is observed from the solution, physically heat is supplied to 
the bottom half of the left-hand boundary and an equal amount is extracted from the 
top half. For comparison of the accuracy, the calculations with h = + and + have been 


6 
included in the table. 


Solve the Poisson equation 
au +2 


=-1 
ax’ dy 


for steady heat conduction with a heating term, in the unit square given that 
(a) du/dx=0 onx=0 (insulated along this boundary); 

(b) w= onx = 1 

(c) u=0 ony =0 (temperature given on three sides). 


(d) uUu=x ony=1 


The region is illustrated in Figure 9.39, with mesh spacing Ax = Ay = } . Equation (9.58) 
just gives u(—1, 7) =u(1, 7) for each 7. Equation (9.55) is modified to take into account 
the right-hand side of the Poisson equation. The value Ax’ is added to the right-hand 
side of (9.55) for each of the interior points. We can write the equations as 


4u(0, 1) = u(-1, 1) + u(1, 1) + 0 + u(0, 2) = 2u(1, 1) + u(0, 2) 
4u(1, 1) = u(0, 1) + u(2, 1) +0 + u(1, 2) + x 
4u(2, 1) =u(1, 1) + u(3, 1) +04 u(2, 2) + 4 
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Figure 9.39 The mesh 
used in Example 9.36. 


i=-l i=0 fs] i=2 i=3 i=4 * 


and so on, and hence obtain 12 equations in 12 unknowns. These can be solved by any 
convenient method to give a solution as shown in Figure 9.40. 


Figure 9.40 Data 
from the solution of 
Example 9.36 using a 


step length 0.25 in each j= 0.2160 0.2630 0.3137 0.3290 0.2500 
direction. ja 0.1329 0.1578 0.1727 0.1567 0.0625 
J=0 0.0000 0.0000 0.0000 0.0000 0.0000 

i=0 i= 1 i=2 i=3 i=4 


9.5.4 Exercises 


55 Use the five-point difference approximation in 


ge (9.55) to solve 


where u(x, 0) = u(0, v) = 0, u(x, 1) =x, u(1, y) 
= y(2 — y). Find the approximations for u(+, $) for 
grid sizes Ax = Ay = } and Ax=Ay=}. 


56 Use a mesh Ax = Ay =} and Ax = Ay = } to 


@) solve 58 


Ju, Jug (<x<1,0<y<]) (2) 
ox oy 

satisfying u(0, vy) = 1, du(1, y)/Ox = 0, u(x, 0) = 0, 

u(x, 1)=1. 


57 A numerical solution is to be determined for the 
loading of a uniform plate, where the displacement 
eB w Satisfies the equation 
2 2 
gw + am +20=0 
ox oy 
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and a square mesh of side h is used. Show that, at 
an interior point 0 with neighbours 1, 2, 3 and 4, the 
approximation to the equation is 


4wy = W, + Wo + W3 + Wy + 2017 


The plate is in the shape of a trapezium whose 
vertices can be represented by the points (0, 0), 

(5, 0), (2, 3) and (0, 3). The plate is held on its edges 
so that on the boundary w= 0. Compute the solution 
for w at the five interior points if is taken as 1. 


The function @(x, y) satisfies the equation 


2 2, 
ox” oy 


and the boundary conditions (see Figure 9.41) 
o=3-y? onOA(x=0,0<y<1) 


gt --4 on AB(y=1,0<x< 1) 
aa 


g=1 on BC (x=1,0Sy <1) 
g=3-x onCO(y=0,0<x< 1) 
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59 


- 
A B 
C 
Oo x 


Figure 9.41 Region for Exercise 58. 


Solve the equation numerically, using a mesh of 


(a) h = § in each direction, (b) h= + 


The function (x, y) satisfies the Laplace equation 


2 2 
7¢ 2% _ 
ox oy 


inside the region shown in Figure 9.42. The 
function @ takes the value = 9x? at all points on the 
boundary. Making full use of symmetry, formulate 


P96 Finite elements 


In Section 9.5.3 we sought numerical solutions of the Laplace equation, but we noted 
that only simple geometries could be handled using finite differences. The region 
described in Exercise 57 is about as difficult as can be treated easily. To adapt methods 
to awkward regions is not easy, so alternative strategies have been sought. Great 
advances took place in the 1960s, when civil engineers pioneered the method of finite 
elements. To solve plate bending problems, they solved the appropriate equations for 
small patches and then ‘stitched’ the latter together to form an overall solution. The job 
is not an easy one, and requires a large amount of arithmetic. It was only when large, fast 
computers became available that the method was viable. This method is now very widely 
used, and forms the basis of most calculations in stress analysis and for many fluid 
flows. It is very adaptable and physically satisfying, but is very difficult to program. 
This is in contrast to finite differences, which are reasonably easy. In general the advice 
to anyone employing this technique is to use a finite-element ‘package’, available in 
most computer libraries, and not to write one’s own program. For instance in the Partial 
Differential Equation Toolbox in MATLAB, finite elements is the standard method of 
solution even for solutions in a rectangular region. As with many toolboxes of this type 
they need a lot of work to master all the details. It is important, however, to understand 
the basis of the method. We shall illustrate this method for a simple situation, but refer 
to specialist books for details and extensions: for example, see P. E. Lewis and J. P. Ward 
The Finite Element Method (Addison-Wesley, Reading, MA, 1991). 
We consider solutions of the Poisson equation 


2 2 
Sat en Pe 


! in each direction. 


-1 


Figure 9.42 Region for Exercise 59. 


a set of finite-difference equations to solve for the 
nodal values of ¢ on a square grid of side h = +. 
Solve for @ at the nodal points. 


(9.59) 
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Figure 9.43 
Triangular finite- 
element mesh, with the 
local numbering of a 
typical element. 


Figure 9.44 

(a) The function Ly; 
(b) wu approximated 
as a linear function 
in the element. 
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in a region R with wu given on the boundary of R. The region R is divided up into a tri- 
angular mesh as in Figure 9.43. We aim to calculate the value of u at the nodal points 
of the mesh, but with the function suitably interpolated in each triangle. The simplest 
situation is obtained if, in a typical triangle, u is approximated as a linear function 


u=ax+by+c (9.60) 


taking the values uw, uw, and w, at the corners. This function can be written explicitly in 
terms of the functions 


Li=lx yw IUfle yw 1 (9.61) 


L,= |X) Yo 1 Xo Yo 1 


x yy 1 xX} 


yi i 
each of which is taken to be zero outside the triangle with vertices (x9, ¥o), (x, y,) and 
(X, ¥). The denominators are just 24, where A is the area of the triangle. The function Lo, 
illustrated in Figure 9.44(a), takes the values | at (%, 9), 0 at (x), y,) and (x, y>); it is 
linear in the triangle and is taken to be zero elsewhere. The functions L, and L, behave 
similarly. The field variable wu in the element, denoted by u°, can now be written as 


U(X, V) = Uply + UL, + WL, (9.62) 
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Example 9.37 


Solution 


The situation is illustrated in Figure 9.44(b). Note that u°(x, y) is a linear function in x 
and y, u°(Xp, Vo) = Up, U(X), ,) = u, and u°(x>, y,) = uy, and hence gives an explicit form 
for the function in (9.60) that has the correct values at the nodes. 


Find the linear function that has the values up at (0, 0), wu, at (1, 1) and uw, at ( i, 1). 


From (9.61), the functions Zy, ZL, and L, are given by 


x y I} /]0 0 1 
L=\|1 1 Wf 1 y=G-swh=1-y 
a ee | a a 
x at 
iye=|\> 1 1 p= 2x-y 
0 0 1 
x y I 
I,=|0 0 1 t= 2y — 2x 
1 1 1 


Thus, from (9.62), the required linear function is 
u= (1 — y)uy + (2x — yu, + 2(y — x) 
or 


u = x(2u, — 2u,) + W(—uy — Uy + 2uy) + Uy 


We build up the solution of (9.59) as the sum over all the elements of the functions 
constructed to be linear in an element and zero outside the element. Thus 


u= yu 


To be of use, this function must satisfy (9.59) in some approximate sense. The 
function cannot be differentiated across the element boundaries, since it has dis- 
continuous behaviour. We therefore have to satisfy the equation in an integrated or 
‘weak’ form. 

We use the well-known result that if V is continuous and 


| | Vodx dy = 0 (9.63) 


R 


for a complete set of functions @ (that is, a set of functions that will approximate any 
continuous function as accurately as desired) then V = 0 in R. Using the residual of 
(9.59) in (9.63) gives 
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Figure 9.45 

(a) A typical node 
and its neighbours 
in the region R; 
(b) the pyramid 
function used at 
the typical node. 
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du , du 
0= (ee +—- Jods dy 
ax ay 


a(40u) , 2(,0u)_ 296 9udd_ lava 
(6 }+ (05) Ber yey D6 il 


= -|fwe, + uo, + po) dx dy — [om dx — gu, dy) 


where the final integral is obtained over the boundary C of R using Green’s theorem 


aN aM - 
\|(2 . oy) ae ay = [ora + Nady) 


R c 


described in Section 3.4.5. 
Choosing @ to be zero on the boundary C, we have an integrated form for (9.59) as 


0= | | (u,0, + uyd, + pd) dx dy (9.64) 


R 


It is this integrated or ‘weak’ form of (9.59) that we shall satisfy. We are therefore 
satisfying the equation in a global sense over the whole region. In comparison, finite- 
difference approximations are local to the mesh point. Clearly, we cannot use (9.64) 
with a complete set of functions @, since there must be infinitely many of them. The 
best that can be done is to use N test functions @, (x = 1,..., N)) when there are NV 
interior nodes. There will then be N equations for the N unknowns u, (n= 1,..., N) at 
the node points. As N > , the functions @, must form a complete set, and then the 
weak form of (9.64) will be satisfied identically rather than approximately. The most 
popular set of functions @; is that due to Galerkin, who used the pyramid functions 
illustrated in Figure 9.45. At a typical point 0 with neighbours 1, 2, 3,..., m we have 


1 atnode0 } and piecewise-linear in each 
= 40 atnodes1,2,...,m 


identically zero outside the neighbouring triangles 


of the neighbouring triangles (9.65) 


(a) (b) 
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If there are NV nodes in the mesh then there are N pyramid functions of the type (9.65). 
We substitute each of these functions in turn into (9.64) to satisfy the weak form of our 
original Poisson equation. Taking a typical node, we see that @ is piecewise-linear in the 
neighbouring triangles. For a typical such triangle 012, @ is just the linear function ZL, 
defined earlier. We substitute @ = Ly into the right-hand side of (9.64) and use the fact that 


U=Uply + uyL, + uly 


to obtain the contribution from this particular triangle as 
Ly ay pa oa) Ss AL 
L= —+u +1 
j || (« dx 0x ax) ax 


Ly OL, Ja OL 
+ LO ars +1 oF re. o|dx dy 


where p, is taken to be constant in the triangle. Since L, (i = 0, 1, 2) are linear, dL,/Ox 
and L,/dy are constants, and hence the integrals can be performed explicitly, giving 


T= {uli —y,P + (xX, - x))] + U[(¥2 — Vo M1 = Vo) + Xz = Xo)(Xy — Xp)] 
+ Ul( Vo — ViVi = V2) + (Xo = XO = Xp) }/4A + +Ap, 


From (9.64) with @ chosen as (9.65), we obtain for the point 0 the sum of such terms 
over neighbouring elements: 


YL=0 


This is just an equation of the form 


m 
> au;+b=0 
i=0 


where the coefficients a; and b depend only on the geometry and not on the field 
variables. 

A similar computation is performed for each internal node, with @ taken to be of the 
form (9.65). The Poisson equation is linear in the u;, and since there is one such equation 
for each internal node, we obtain N equations in the N unknowns u; (i= 1,..., N). These 
form a matrix (called the stiffness matrix) equation, which can be solved for the u;. The 
general strategy is 


(i) calculate all the coefficients; 

(ii) assemble the stiffness matrix; 

(iii) invert the matrix to obtain the unknowns u; (i= 1, 2, 3,..., N); 
(iv) calculate any required data from the solution. 


For the Laplace equation with linear approximating functions in triangular elements, 
a MATLAB implementation will be developed. In this development, note the com- 
plexity of the programming even for this very simple situation and also the organi- 
zation of the input data required by the program. There has been no attempt at 
efficiency in the programming of the sections of the code. 
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Example 9.38 


Figure 9.46 

Mesh for the finite- 
element solution to 
Example 9.38. 


Solution 
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Solve the Laplace equation @’u/dx? + Pu/dy* = 0 in the unit square shown in Figure 9.46, 
subject to the boundary conditions indicated. 


Here we have a simple rectangular region with p = 0, and which is the same problem 
as Example 9.34. 


A012 givesacontribution 0.625u) — 0.375u, — 0.250u, 


A023 n 0.500uy — 0.250u, — 0.250u, 
A034 n 0.625uy — 0.250u3 — 0.37514 
A045 n 0.625uy — 0.375uy — 0.250us 
A056 n 0.5000) — 0.250u5 — 0.2501, 
A061 n 0.625uy — 0.250u, — 0.3751, 


Adding all these contributions for the point 0, which is the only unspecified point, gives 
0 = 3.5uy) — 0.75u, — 0.50u, — 0.50u, — 0.75u, — 0.50u; — 0.50u, 


so that, knowing uw, = uv) = u, = us = Uy = 0 and u, = 1, we obtain uy = 0.2143. Comparing 
with Example 9.34, we see that our result is not particularly accurate, which is not 
surprising, since the mesh chosen here is particularly crude. 


It is clear from Example 9.38 that the contributions from each triangle need con- 
siderable computational effort and the finite-element method is unsuitable for hand 
computations. 


The coefficients in Example 9.38 can be computed from the MATLAB M.-file stored 

under the name coeff:m. The coordinates of the vertices of the triangle are inserted as 

a=[p q],b=[r,s] and c=[u,v]. The coefficients are produced in a0,al,a2. 
function[a0,al,a2|\=coeffi (a,b,c) 
Asle Ii; Bells ily Cele ti, belli © Ole IbvelG 1 Ol; 
den=0.5/det ([A;B;C]); 

LO= lelete (eee isgiCl))) che (lly yeeCl) iy 

ihi= |[olete (\Ibzg Cell) Clete ( [sy p@ Aull ) 1 ¢ 

L2=[det ([1*x;A;B]) det(ily;A;Bl)1; 

AUSLO°™LOwceny ili’ LO-Cleing s2eb2 "iL Oclems 


Such a function file will be used in a more general program. 
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Example 9.39 


Figure 9.47 Mesh for 
Example 9.39. The 
unmarked boundary 
points are given as 


u=0. 


Solution 


Solve the Laplace equation in the region shown in Figure 9.47 subject to zero boundary 
conditions except for the three points indicated. All the triangles are equilateral of side a. 


u=1/2 


oe 
ae 


Note that in the region in Figure 9.47 it would be very difficult to implement a standard 
finite-difference mesh. We now follow the general strategy. 


(i) 


(ii) 


Calculate the coefficients. When all the triangles are equilateral, the coefficients are 
all identical so the amount of computation is greatly reduced. For a typical point 


A012 givesacontribution (4) — 2u, — 2u,)/4/3 
A023 " (4) — 2u, — 2u;)/4./3 
A034 " (4ui) — 2u, — 2u,)/4./3 


and hence adding the six contributions gives the total for the typical point 


(6) — Uy — Uy — Uy — Uy — Us — Ue)/\3 

Assemble the stiffness matrix. Apply the results in (i) to each of the six active points 
6u, =u; +uUytu,+0+ ; +1 

6u, =u,+u,+0+04+0+4+u, 

6u,=0+ ug tug tu, + ; +1 

6u,=Ug + O+uUs+u,+u, +U; 

6u;=0+04+04+04+m4+uy 

6u,=04+04+04+u,+u,+0 


and the matrices take the form 


Ss teat at oo [3/2 rn 
-l 6 0 -1 -1 O 0 U> 
A- -l 0 6-1 0 -1 b= 3/2 wal ® 
-1 -1 -l 6 -1 -l 0 U4 
0 -l O -1 6 O 0 Us 
0 oO -1 -1 0 6 0 Us 
: stiffness matrix - load vector unknowns 
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(iii) We now need to solve the matrix equation Au = b for the vector u. It may be 
noted that the matrix does not have much ‘structure’, except that it is diagonally 
dominant, so a direct inversion is usually preferred unless the dimension of the 
matrix is very large. The equations were solved using MATLAB to give 


0.3481 | 
0.0900 
0.3471 
0.1514 
0.0402 
| 0.0831 | 


(iv) Calculate any required data from the solution in (111). 


To construct the rows of the stiffness matrix, A, ina MATLAB implementation, the 
coordinates of the nodes are put into a matrix 


x) Vi 
CO= 22 


Xn Yn 


with the internal nodes first followed by the boundary nodes (note in the MATLAB 
program this matrix is declared as global). The neighbours of each of the internal 
nodes are placed in the matrix Jink, one row for each node. The MATLAB function, 
stored in the M-file stiff'm, computes the contribution to the stiffness matrix from 
the Ath internal point, the output a gives the contribution to the kth row in the full 
stiffness matrix. 


function a=stiff(mm,k,L) 
gmm=no of neighbours, k=current point, L=row of k’s 
neighbours 
global CO 
a=zeros(1,mm+1); 
EO jos smmim iL 
[1 tit, Tal] COSINE (COM, 8) pCO) 5 §) pCOME (eel) 7 3) ) a 
® note that coeff.m is used 
a(1)=a(1)+1; a(p+1)=a(p+1)4+m; a(p+2)=a(p+2)+n; 
end 
[1 im, mil] coerce (CO(, 8) COL (mm) , 2) ,COUL(1) , 2) is 
a(1)=a(1)+1; a(mm+1)=a(mm+1)+m; a(2)=a(2)+n; 
The following example illustrates the use of MATLAB in the solution of the Laplace 
equation. 
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Example 9.40 — Solve the Laplace equation V*@ = 0 in the elliptical region 
2 
7 3 + —— =1 
cosh'l  sinh'l 
with = 1 on the upper half of the ellipse and @ = 0 on the lower half. The situation is 
illustrated in Figure 9.48. 


Figure 9.48 Mesh for 1.5 1 1 1 1 1 T 1 
the elliptical plate in 
Example 9.40. 
TF 4 
0.55 4 
y OF 4 
-0.5- 4 
—-l- 4 
1.5 1 1 1 1 1 1 1 
2 15 1 0.5 0 0.5 1 1 2 


Solution The problem corresponds physically to an elliptical plate, hot on one side and cold on 
the other. In Figure 9.48, the triangulated mesh illustrated is the one used in the program. 
Note how well the mesh fits the boundary even for a small number of boundary nodes. 

Data for the problem is placed in a script file stored as inform.m. 


(2) nin=5; nbdry=12; %number of internal and boundary nodes 
@llolseill CO 
WHO Sidi /S9S io /Ge x= Cosin (il) "cos (y") , sSaliala (41) sssiia (y") I 


iS (c(i, 8) ee (2 8 eee (3, osx, sy) Wis 
COSIL0) Og =i (al, ih) SLL, 2) paxil (4, il) seal (al, 2) ge xeal (GL, al) 
=A (AL, 2)) Roxsd 3x || 3 

Scoords of points, internal first then bdry 
lim(2 3 15 425 9 1 GO 10 di 12 Sp l 2 ie 13 24 ass i 15 
16 iy 6 Se 1 46 7 8 Qs 

S$links from interior points to neighbours, in CO order 
bdry=[0.5000000.511111]; %boundary values, in CO order 
Sewers (Galina) p iclagswzeios (imain.,, iL) 2 


The solution is then computed from the following code, which should be stored in some 


convenient place so that it can be edited easily. The complete stiffness matrix, A, is 
assembled and all the boundary data is transferred to the right-hand side vector called rhs. 
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Example 9.41 


inform %inserts the data from inform.m 
for k=l:nin %for each internal node transfers info to A or 
rhs 
iellaalk (Ik, 8)) 2 iin=ietay (Ge) 9 
MSC IEE (iid, Ne, Te) 9 
uses stiff.m, which in turn uses coeff.m, to compute the 
contributions from row k 
A(k, k)=A(k,k) +z (1); 
ewe aleil Simm 
ie ie (Simla 
INE, ie (aL) Sv (Te, 12 (aL) see (ate 1L}) 5 
else 
rhs (k) =rhs (k) -z(i+1) *bdry (r(1)-nin) ; 
end 
end 
end 
A, rhs, A\rhs %prints out the stiffness matrix, the rhs 
and the final solution 


The print-out is 


A= 4.3857 -1.1266 -1.1266 -1.1266 -1.1266 rhs= -0.0604 
-1.1266 3.9085 -0.6836 0 0) 0.0699 
-1.1266 -0.6836 3.9085 0 0 2.0284 
=1,.1266 0 0 3.9085 -0.6836 2.0284 
-1.1266 0 0 -0.6836 3.9085 0.0699 


which gives the final solution as 
0.5000 0.2868 0.7132 On 7132 0.2868 


The solution has all the correct symmetries about the x and y axes. An exact solution to 
the problem can be obtained by separation of variables in an appropriate coordinate 
system in terms of Fourier series. For node 3 this method gives the value 0.7076 com- 
pared with the FE value of 0.7132; an accuracy of less than 1% has been obtained. 


For the solution of the Poisson equation with p # 0 in (9.59) all the segments of the 
MATLAB programs need to be modified. A similar problem in a rectangular region 
was studied in Example 9.36 using finite differences. 


Solve the Poisson equation 


Tu + Ju ==) 
ox” oy 

in the hexagonal region illustrated in Figure 9.49 and with uw = 0 on the boundary of 

the region. 
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Figure 9.49 
Hexagonal region 
with mesh used in 
Example 9.41. 


Figure 9.50 
(a) Flow in 
hexagonal pipe; 
(b) cylinder in 
torsion. 


Solution 


1 it 1 if i 1 i i if 1 J 
1 0.8 -06 -0.4 -0.2 02 04 06 08 1 
x 

7 

~~ ute, y) i. 

@ : 

ORS gripped and 
sa i twisted 
(b) 


There are several physical interpretations of this problem. In the context of heat flow, the 
boundary is kept at a fixed temperature and heat supplied at a uniform rate to the plate. 
For the unidirectional flow of a viscous fluid in a long hexagonal pipe, u is the velocity 
and the constant right-hand side is related to the pressure gradient along the tube, see 
Figure 9.50(a). A honeycomb of tubes in a heat exchanger is a possible application. 
When a cylinder is in torsion, by gripping at one end and gripping and twisting at 
the other, the stresses can be computed from the same Poisson equation; for an illus- 
tration see Figure 9.50(b). For a detailed description of these physical problems and the 
derivations a specialist book should be consulted (S. C. Hunter, Mechanics of Con- 
tinuous Media, Ellis Horwood, Chichester, 1983). 
The modifications to the MATLAB implementations can be checked easily against 
the same problem with an elliptical region since an exact solution is known to be 


242 2 2 
= $2 (1-4-4) 
a eh a b 


for the region 


cae 
a 
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The stiffness matrix is the same as in Example 9.40 and only the right-hand sides are modified. 
For the hexagonal region with the mesh shown in Figure 9.49 the matrices are com- 
puted using the modified MATLAB segments and the print-out is 


A= 3.4641 -0.5774 -0.5774 -0.5774 -0.5774 -0.5774 -0.5774 


-0.5774 3.4641 -0.5774 0 0 0 -0.5774 
-0.5774 -0.5774 3.4641 -0.5774 0 0) 0) 
-0.5774 0 -0.5774 3.4641 -0.5774 0) 0) 
-0.5774 0 0 -0.5774 3.4641 -0.5774 0 
-0.5774 0 0 0 -0.5774 3.4641 -0.5774 
-0.5774 -0.5774 0 0 0 -0.5774 3.4641 


rhs’ = 0.4330 0.4330 0.4330 0.4330 0.4330 0.4330 0.4330 
and gives the solution 
0.4167 0.2917 0.2917 0.2917 0.2917 0.2917 0.2917 


The problem has a great deal of symmetry which is reflected in the solution and, 
indeed, symmetries could have been built into the program to reduce the computational 
effort. In the current situation the solution is computed almost instantly, but in most 
engineering problems every bit of symmetry should be used to its fullest. 


In this section no more than the ‘flavour’ of the finite-element method has been given. 
The intimate connection with computers makes it difficult to do more than show the 
complications that occur and give an outline of how they are dealt with. 

For many problems a linear approximation is not good enough, for example in stress 
analysis for finite deformations, when high derivatives are required. Also, there is no 
good reason why a triangle is chosen rather than a quadrilateral. There are several 
choices that must be made at the start of the calculation: 


(i) division of the region into triangles, quadrilaterals, . . . ; 
(ii) level of approximation, linear, quadratics, cubics, ... ; 
(iii) choice of test function; 

(iv) method of integration over elements: exact, Gaussian, ... ; 
(v) method of labelling nodes; 

(vi) method of solution of the resulting matrix equation. 


As indicated in (iv), once we have abandoned linear approximations, the integrals 
cannot be performed exactly, and we need to use an approximate method. Gaussian 
integration for triangles works very well, and is commonly used. Usually, there is no 
obvious labelling of nodes and it is necessary for each node to keep a list of which 
nodes are neighbours, as in the matrix link in the MATLAB segment inform.m. In the 
Partial Differential Equation Toolbox in MATLAB an automatic triangulation of a region 
can be found; it automatically chooses the nodes and their coordinates, the labelling, 
as in (v), and the list of neighbours. The complexity can be appreciated for a simple 
rectangular mesh from the MATLAB commands 

g='squareg’; 
[p,e,t]=poimesh(g,4); % p,e,t represent the points, edges, 


triangles respectively 
pdemesh(p,e,t) 
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60 


61 


Because the labelling is not straightforward, the resulting stiffness matrix rarely has a 
simple structure, and the most usual method of inversion is by a full frontal attack 


with Gaussian elimination. 


The method has been illustrated for only one equation, the Poisson equation. A similar 
analysis has to be undertaken for each new equation considered. 


9.6.1 Exercises 


All these exercises require substantial programming expertise. Alternatively the Partial Differential Equations 


Toolbox in MATLAB can be used. 


Solve the Laplace equation for the rectangular 
region0 Sx S6and0 Sy S 2,3 using 

finite elements. On the right hand boundary u = 1 
and zero on the remainder of the boundary. 


(a) Use the mesh in Figure 9.51(a) with two 
interior points. 

(b) Use the mesh in Figure 9.51(b) with five 
interior points. 


Figure 9.51 Mesh for Exercise 60: (a) with two interior points; (b) with five interior points. 


Solve the problem in Exercise 57 using the 
triangular finite-element mesh shown in 
Figure 9.52. 


233 
(0,3) (2, 3) 


(0, 0) (5, 0) 


Figure 9.52 Finite-element mesh for 
Exercise 61. 


Solve the problem in Exercise 59 using the triangular 
finite-element mesh shown in Figure 9.53. 


Figure 9.53 Finite-element mesh for Exercise 62. 
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Integral solutions 


In previous sections solutions were built up from elementary solutions of various partial 
differential equations, the most obvious of which was from separated solutions. There 
are many problems where separated solutions are not available but building up from 
elementary solutions may still be possible. This section will show methods of solution 
which can lead to very important ideas that can be exploited practically and importantly 
to some proofs of existence and uniqueness of solutions. Some numerical methods also 
use these ideas, for instance the boundary element method is an extension of finite 
elements with the advantage that the dimension of the calculations is reduced by one. 
On the whole the mathematics is quite demanding so the interested reader is left to 
explore the full power of the new methods in specialist books. 


Separated solutions 
In plane polar coordinates with x =r cos 0, y=r sin 0, the Laplace equation 


ou 47u _9 


ax’ ay” 7 


becomes (see Example 3.6 in Section 3.1.1) 


Lat) 3) ow 


Writing u = F(r)G(@) and substituting gives the separated equations 

1,.d(,d6) soli Gg 

F dr\ dr Gdé 
or 

2 
ro ree) =F and $946 =0 
dé 

For periodic solutions of the equation in G the constant 4 must be an integer n. Thus 
the solution is 

G=AcosnO0+B sinn@ 


and for the F equation it follows easily that 


race 
r" 


where A, B, C, D are arbitrary constants. We now choose a specific problem to illustrate 
the use of these solutions. 


Solve the Laplace equation V’u = 0 inside the circle r = a with u specified on the bound- 
ary as u(a, 0) = f(0), a continuous periodic function with period 27. 


If the solution is finite inside the circle then F' must be of the form F = Cr”. A sum of 
all the terms then becomes 
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u(r, 6) = Ao + ¥r"(A, cos n6 +B, sinh n6) (9.67) 


n=1 


and the solution is now a standard Fourier series problem (see Chapter 7) namely 


u(a, 6) =f(@) = 54 + ¥a"(A, cos n6 +B, sin n6) 
n=l 
The coefficients are 


20 20 


a'A, => | f(f) cos(nt) dt and B,=+ | f(t) sin(nt) dt 
0 0 


These coefficients are substituted into (9.67) and the summation and integration 
are interchanged; this is permissible since f(@) satisfies the Dirichlet conditions (see 
Section 7.2.9) 


20 
l1iwf(ry . . 
u(r, 0) = 1] 19 a), () (cos n@ cos nt +sinn@ saan 
2 a 
0 L 
or 


u(r, 0) = - | fi) ; +> (£) cos n(0- J dt (9.68) 


n=1 


0 L 


Now consider the series 


stztzez +... where z= Re!? 


which has the sum, for |z| < 1, 


1 z _ ltz 


0 ee Bl =z) 


Take the real part and we obtain, after a little algebra 
1=R 
2d -2R cos @ +R ) 


Use this expression in (9.68) to obtain a final result 


5 + Ros o+R> cos2o+.. 


20 


u(r, 0) = +| tox 


0 


2 
=F 


a en (9.69) 
a -2ra cos(@ - f) +r 


which is called the Poisson integral formula. 


Example 9.42 shows that the solution of a complicated partial differential 
equation can be reduced to an integral. The problem has been reduced essentially from 
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a two-dimensional problem to a one-dimensional problem on the boundary. Integrals 
are well understood and there is a vast array of methods that can be used to solve the 
problem either explicitly or numerically. In general if a method can be developed to 
convert a partial differential equation to an integral form on the boundary then a great 
deal of progress has been made to obtaining a solution. 

Not least of the advantages of an integral formulation is that bounds on integrals are 
much easier to obtain than on differentials. These are used almost exclusively to prove 
uniqueness and existence of solutions. A result that follows easily from (9.69) is 
obtained by putting r = 0 

20 

mo j= | f(t) dt (9.70) 
27 

0 
so that the value at the centre of the circle is the average of the values on the bounding 
circle. Thus the value at the centre of the circle can never be the maximum (or minimum) 
value in the region. For a general Laplace equation problem, since every interior point can 
be placed at the centre of a small circle, it can never be the maximum. We can therefore 
deduce that for the Laplace equation the maximum (or minimum) value cannot be in 
the interior but must be on the boundary. This result is one of the keystones of the proof 
of uniqueness of solution. 


Use of singular solutions 


Again consider the two-dimensional Laplace equation (9.66) in plane polar coordinates. 
If we look for solutions, f(7), that only depend on r, then the equation becomes 


Ld (dt) 9 
rdr\ dr 


which can be integrated as 
rat =A andthen f=Alnr+B 
a 


where A, B are arbitrary constants. The solution has a singularity at the origin which 
can be exploited to obtain more general solutions and reduce the problem again to an 
integral round the boundary, as in Section 9.7.1. The method is based on Green’s the- 
orem discussed in Section 3.4.5 


pars Ody) = \| & 7 =) dxdy (9.71) 


Cc S 


where the curve C encloses the region S. Put 


ov , ou Ov. ou 
P=-uS +v> =uS-vpS 
ee ey and QO un vu. 


v4, _ O4\_ (Aus, dus \ _ 2? ; 
u(y sar) »( Hay ax) [fw vV'u) dxdy 


Cc Ss 
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Figure 9.54 Region S 
bounded by the curve 

C, ‘punctured’ by the 

small circle C’. 


(«2 vSt\as = || (wV?v -vV2u) dxdy (9.72) 
on on 
Ss 


where 7 is the normal direction and s is the length parameter along C. 

We now need to choose wu and v in the extended Green’s theorem (9.72) to obtain 
useful results. The region considered is the interior of S, which is bounded by the curve 
C, ‘punctured’ by a small circle C’ with centre ry and radius € (which we will eventually 
tend to zero), r= ry + €(cos @, sin ) (see Figure 9.54). Take 


1 
=——InIr— 
u an n|jr-7l 


and consider the second term in (9.72) on the small circle C’ 


ou ou_ loa 11 
— = ——| = = 
on’ Og 2ndE a 2nME and’ as e0e 


Thus the term becomes 


Bide He aha wo Te stag 
oda] peed? H.| vue V(T) 
0 0 


Cc’ 
The last result just comes from (9.70) and we see that this term just picks out the 
value of v at the point ry. It remains to construct appropriate u and v to exploit this 
idea. It is left to specialist texts to consider the choices for general problems; here we 
will concentrate on the Dirichlet problem (Section 9.8.2) for the Poisson equation, 
where 


V’w=-p(x, y)_ in the region S and y given on C 
and let 


al 


5, mle — rol + AG, Ys Xo, Yo) (9.73) 
Tl 


Wy’ = G(X, ¥; Xo, Yo) = 
so that 
V’y’ =0_ in the region S and yw’ = 0 on C 


and H has no singularities in the region. In (9.72) put u = yw’ and v = w. Because y’ 
satisfies the Laplace equation and y the Poisson equation in the region S we can write 
(9.72) as 
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,OYW _ | _ OW _ g(t _ } 
(v on ue ds ply > Vo. Sylar rol +H(r, ro) |/e de 
Cc Cc’ 


=-|| crave 
S 


Taking the left-hand side terms one by one: the first term is zero since yw’ = 0 from the 
conditions given; the second gives the required integral round the boundary; the third 
term is of order € In € so tends to zero as € > 0; the fourth term was treated above and 
gives y(ro); and the fifth term is of order € and tends to zero. Collecting up the terms 
gives 


W(X; Yo) = pues y) 2 G(x, ys XO» Yo) ds ot [Joc J; Xo» Yo) Px, y) dx dy (9.74) 
€ 


We can now find y at any point in the region from the value of yon the boundary, the 
right-hand side of the Poisson equation p(x, y), together with the function G(x, y; Xo, Vo), 
called Green’s function. At the moment it is assumed that Green’s function exists and 
can be calculated. For simple geometries it can often be found and advanced books 
show how and when this can be done. The whole theory of Green’s functions can be 
applied to many different equations and boundary conditions but this is the province 
of advanced books on partial differential equations (see R. Haberman, Applied Differ- 
ential Equations, Prentice Hall, Upper Saddle River, NJ, 2003). An example will illus- 
trate the method. 


Solve the Laplace equation 
V’f=0_ in the region y > 0 


given that f(x, 0) = F(x), a known function, on the x-axis and that fis zero at infinity. 


Green’s function (9.73) can be constructed by reflection as 
1 1 
G[(X, ¥3 Xo, Yo)] = —== In| (& — Xo, ¥ — Yo) | + == In| (& — Xo, Y + Yo) | 
20 20 


=~ fein Sak HO 
Am ((x- x9) +(y +0) 


Note that the added term has no singularities in the region y > 0; the function is zero 
on y = 0 and tends to zero as x and y tend to infinity. Now 


eG ~.0G = 1 2(y = Yo) 2 2(y +¥o) 
On Oy AR) (x= x9)? #(v- Yo)? (X= Xo)” H(Y +0)” 


Putting y = 0 gives 
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9.7.3 


| ee 
On R] (x x9)” +¥ 


The solution is then obtained from (9.74) as 


( 1 
o> Yo) = | Ft 20] 
(o> Yo) | wl ail 


Again we see that the solution has been reduced to an integral along the boundary. 
Finding the exact form of Green’s function is known for some classic problems, like 
the one in Example 9.43, but in general it is a difficult calculation. The problem is 
closely connected to finding a solution of the partial differential equation with zero 
boundary values (at least for the Dirichlet problem) and a Dirac delta function imposed 
at a point in the interior. 


Sources and sinks for the heat conduction equation 


In Example 9.4 a solution to the one-dimensional heat-conduction equation was 
obtained which corresponded to a pulse of heat being supplied at a particular point at zero 
time. The subsequent dissipation of the heat pulse is illustrated in Figure 9.4. A similar 
solution can be obtained for the three-dimensional problem. The radial symmetric equation 
of the heat-conduction equation is 


oT _ i al 20T } 
==K= =r = 9.75 

ot or "Or Ce) 
where r is the radial distance from the origin and k= k/( pc) is the thermal diffusivity. 
The parameter k is the thermal conductivity, p is the density of the medium and c is the 
specific heat capacity. The solution corresponding to the one-dimensional solution of 
Example 9.4 is 


Q/(pc) r 
_ ma 9.76 
8(mKt) exn( -) Ce) 


The solution can be verified by direct substitution, as in Example 9.4. It is noted that 
there is a singularity at zero time which corresponds to a point source releasing an 
instantaneous amount of heat Q, calculated as follows: 

The total amount of heat in the whole of the space at time ¢ is computed from the 
amount of heat in the shell of radius r and thickness dr and then integrating 


2s Nae z g Yap 
H= [om *(pcT) dr = [snr 2 —exo(- 
0 0 


The integration can be obtained by differentiating with respect to a the well known integral 


co Co 


-az" 1 TU 2 -az* 1 TU 
dz= 5 /E to get dz= 7/4 
|: o=G 0 ge [Fe aa 3 


0 0 
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Thus 


H=—2 — Fann” =9 
2\n(xty” 4 


The solution (9.76) is usually called an instantaneous source, releasing an amount 
of heat Q at time zero. As expected for fixed ¢ and as r tends to infinity the temperature 
T tends to its resting temperature of zero. Also for fixed r the temperature tends to zero 
as the time ¢ tends to infinity and all the heat is conducted away. 

For a continuous source, heat is released continuously at some given rate. In a time 
interval ds at time s, assume that heat released is g(s)ds then the temperature is given 
by (9.76) with the time starting at s. Temperature due to release of heat g(s)ds at time 
s is given by 


_4Us\i(pe) (-pos pee 
8(mK(t-s))” 4«(t-s) 


Thus over the whole interval then 


t 


_ 1 qs) r 
DS -————- |d 9.77 
eo 8pc(mK)” | ao. exp( mars) 7 eo 


0 


Again we see that the solution of the heat-conduction equation can be written as an 
integral with all the advantages listed earlier. For most functions q(s), the integral in 
(9.77) cannot be performed explicitly, but for g = go, a constant, it is possible. Making 
the substitution 


u=ri[4K(t—s)]\? 


reduces the integral to 


co 


ae | exp(-u’) 2(4x)'"2du 
Cc(uk 
p r/\ (4Kt) 
ae eee “i)du= de Lerfe{ —t—) 9.78 
2pckn’r | a 4pcxnr \(4xt) Oe) 
rly (4Kt) 


where erfc is the known function defined in Example 9.24 and can be found in all 
computer packages. 

It can be seen that for large times the erfc function tends to one so the steady tem- 
perature due to the source decays like 1/r and the steady temperature T, is 


(7) = —t0_< (9.79) 


This function must satisfy the Laplace equation; except at the origin, and gives the 
three-dimensional singular solution that is used to construct Green’s function in an 
exactly similar manner to the two-dimensional version described in Section 9.7.2. 

Many situations can be tackled using (9.76)—(9.79) and the solution can be reduced 
to an integral, which usually requires a numerical quadrature. 
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Example 9.44 


Solution 


Figure 9.55 

(a) Line source 

and (b) cylindrical 
coordinates (R, z) for 
Example 9.44. 


Find the steady temperature due a constant line source of length 2Z, placed in an infinite 
conducting medium with constant thermal properties. 


We will use the steady point solution in (9.79). The axes are set up in Figure 9.55, with 
(R, z) being the cylindrical polar coordinates of the field point relative to the origin at 
the centre of the rod — note that there is no angular coordinate since the solution is 
clearly symmetrical about the z axis. 


(a) 


L (R, 2) 
z—p 


>R 


Take an element of the line at (p, 0) of length dp releasing heat at a rate of g,dp then 
from (9.79) the temperature at (R, z) due to this element is 
vis 1 
—_ ———_ dp 
4pcK |R? +(z- py 
Hence the temperature due the whole of the line is 


L 


vi 1 
T,(R, Zz) = —— dd 
1(R, 2) 4pcKn lz sean P 


-L 


__ 4 aw(2 +1) wae (: : 4) 
4pcKkn R R 


Such a calculation can be used to model the temperature due to a heated pipe or 
cable buried underground or diffusion of contaminant from a section of a steadily 
leaking pipe. The effect of the burial of a line source at a distance below a surface with 
a fixed temperature can be calculated by adding a parallel line sink at an equal distance 
above the surface (see Exercise 66). Note that for large R the square bracket behaves 
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like 2L/R to first order so that at large distances the line source just looks like a point 
source with strength 2Lg,. A wide range of applications of these ideas can be found in 
H. S. Carslaw and J. C. Jaeger, Conduction of Heat in Solids, Oxford University Press, 


New York, 1959. 


9.7.4 Exercises 


63 Use the Poisson formula (9.69) to solve the Laplace 
equation in the disk r S a with the temperature given 
as T= T, for0 < @< mand T=0 form < 0< 2n, 
where (7, 9) are plane polar coordinates. 


64 Show that 
u(x, y) =  In[(x - a? + (y - bY] 
4n 


satisfies the Laplace equation at all points except 
(a, b). Check that the function 


G(x, V3 Xo. Vo) = 


4m [x= xo) + Hye WO +0) +0-¥e 9ge 


satisfies all the properties of the Green’s function 
for the Dirichlet problem for the Laplace equation 
in the quarter region x = 0, y = 0. Hence solve 
the Laplace equation V’7 = 0 in the region x = 0, 
y = 0 with the boundary conditions T(x, 0) = f(x) 
for x = 0 and 7(0, y) = g(y) for y = 0 and T remains 
bounded at infinity. Show that 


T (Xo ¥o) = 


| {fe - la ie 
TJ Ue-x) ty (x +x) ty 


je dy 


TI} Uy ty) +x5 (0) FXG 
i Y TYo 0 ¥~ Yo 0 


Evaluate T when f(x) = 1 and g(y) = 0. 


65 — Green’s function of the Dirichlet problem for the 
Laplace equation in the disk, y S a, can be written in 
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terms of polar coordinates of the point (7, @)) and 
its inverse point (a’/rp, @)). Check that the function 


Gr, 6; La 6) a 


a 2ra’ 
ae —— cos(0- 0) 
1 jjjm to 7 


4n jar +r, - 2rr)cos(@- 05) 


2: 
r 


satisfies the conditions of Green’s function 

with G = 0 on r) =a. Deduce that the solution 

of the Laplace equation in the region r S a and 

u(a, 0) = f(@) is given by the Poisson formula (9.69). 


Find the steady temperature 7(x, y, z) due to a 
constant line source of length 2L, placed at x = a, 
y=0,-L Sz SL with the plane x = 0 maintained at 
zero temperature. Use the result in Example 9.44. 


A uniform ring source consists of instantaneous point 
sources at the points of the circle z= 0, 2° +)" =a or 
x=acos 0, v=asin @. Each element of the ring, ad6, 
releases an amount of heat gad@ at time t= 0. Use 
(9.76) to show that the temperature at any point 

(R cos @, R sin @, z) is 


T(R cos @, R sin @, z) 

_ __q2na ( Ro +2 — ($4) 

= tt exp| -+— I, — 
8pc(mKt)” 4kt 2Kt 


where J, is a modified Bessel function, which is a 
known function available in MAPLE and MATLAB. 
It can be defined as 


2n 


[)(@) = x | exp(@ cos y) dy 


0 
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Sex) General considerations 


9.8.1 


There are properties of a general nature that can be deduced without reference to any 
particular partial differential equation. The formal classification of second-order equa- 
tions and their intimate connection with the appropriateness of boundary conditions 
will be considered in this section. The much more difficult problems of the existence 
and uniqueness are left to specialist texts. 


Formal classification 


In the preceding sections we have discussed in general terms the three classic partial dif- 
ferential equations. We shall now show that second-order linear equations can be reduced 
to one of these three types. 

Consider the general form of a second-order equation: 


Au,, + 2Bu,, + Cu,, + Du, + Eu, + F = 0 (9.80) 
where A, B, C,... are constants. If we make a change of variable 
r=axty, s=xtby 


then the chain rule gives 
Uy, = @U,, + 2au,, + Ug, 
U,, = au,, + (1 + ab)u,, + bu,, 
Uy, = U,, + 2bu,, + b’u,, 
Substituting into (9.80) gives 
u,(Aa@ + 2Ba+ C)+ 2u,(ad + B+ abB+bC)+u,(A + 2Bb+ b’C) 
+ (aD + E)u,+ (D+ Eb)ju,+ F=0 
If we choose to eliminate the w,, term then we must put 
a(A + bB)=-(B+bC) (9.81) 
and we can eliminate a by substitution to obtain 


2 
(4 +2bB + B°C)| u,, + AF | +... =0 (9.82) 
(A + Bb) 


We can see immediately that the behaviour of (9.82) depends critically on the sign of 
AC — B’ and this leads to the following classification. 


Case 1: AC — B’ > 0, elliptic equations 

On putting (AC — B’)/(A + Bby = #, (9.82) becomes 
a(u,,+ Lu,,)+...=0 

and on further putting g = r/A, 
U,, tg +... =0 


The second-order terms are just the same as the Laplace operator. Equations such as 
(9.80) with AC — B* > 0 are called elliptic equations. 
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Solution 
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Case 2: AC — B* = 0, parabolic equations 
In this case (9.82) simply becomes 
U,+...=0 


with only one second-order term surviving. The equation is almost identical to the heat- 
conduction equation. Equations such as (9.80) with AC — B? = 0 are called parabolic 
equations. 


Case 3: AC — B’ < 0, hyperbolic equations 
On putting (AC — B’)/(A + BbyY = -10, (9.82) becomes 
O(u,, — [eu,,) +... =0 
and on further putting ¢ = 7/u, 
Uy —Uyt...=0 
which we can identify with the terms of the wave equation. Equations such as (9.80) with 


AC — B’ < O are called hyperbolic equations. 


Thus we see that simply by changing axes and adjusting length scales, the general equa- 
tion (9.80) is reduced to one of the three standard types. We therefore have strong 
reasons for studying the three classical equations very closely. An example illustrates 
the process. 


Discuss the behaviour of the equation 
Uy, + 2U,, + 20U,, = 0 


for various values of the constant a. 


In the notation of (9.80), A = 1, B= 1 and C= 2a, so from (9.81) 


_ _1+2ab 
1+b 


and the change of variables r = ax + y, s=x + by gives 
2a—-1 = 
= 
(1 + b) 


u SS 


Thus if @ > $ and qg=r(1 + b)/\(2a@— 1), we have the elliptic equation 
Us, + Ugg = O 

Ifa= s, we have the parabolic equation 
Uy, = 0 

Ifa< ; and t= r(1 + b)/\(1 — 2), we have the hyperbolic equation 


Us, — Up = 0 
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9.8.2 


In (9.80) the assumption was that 4, B, C, ... were constants. Certainly for many 
problems this is not the case, and A, B, ... are functions of x and y, and possibly wu also. 
Therefore the analysis described may not hold globally for variable-coefficient equa- 
tions. However, we can follow the same analysis at each point of the region under 
consideration. If the equation at every point is of one type, say elliptic, then we call the 
equation elliptic. There are good physical problems where its type can change. One of 
the best known examples is for transonic flow, where the equation is of the form 


2 2 
(1 ~ i ~ ait + (: ~ LY» ot y) =0 
Cc Cc c 


where w and v are the velocity components and c is a constant. We calculate 


ue y uv \ uty é 
2 = ~-, 4274 _;,_4 
AC-—B =(1- 5 \(1- ;} 3) = 1 es 1 ; 


c c Cc Cc 


If we put g/c = M, the Mach number, then for // > 1 the flow is hyperbolic and supersonic, 
while for M < | it is elliptic and subsonic. It is easy to appreciate that transonic flows 
are very difficult to compute, since different boundary conditions and techniques are 
required on the subsonic and supersonic sides. 


Boundary conditions 


In the preceding sections we chose natural boundary conditions for the three classical 
partial differential equations. We can formalize these ideas a bit further and look at 
appropriate boundary conditions and the consequences of choosing inappropriate 
conditions. We shall confine ourselves to two-variable situations, but it is possible to 
extend the theory to problems with more variables. 

Suppose that we are trying to obtain the solution u(x, y) to a partial differential equation 
in a region R with boundary C. The commonest boundary conditions involve u or the 
normal derivative Ou/dn on C. The normal derivative (which is discussed in Section 3.2.1) 
at a point P on C is the rate of change of wu with respect to the variable n along the line 
that is normal to C at P. The three conditions that are found to occur most regularly are 


Cauchy conditions 
u and ou given on C 
on 


Dirichlet conditions 
u given on C 
Neumann conditions 

Ou 


— given on C 
on e 


It is common for different conditions to apply to different parts of the boundary C. A 
boundary is said to be closed if conditions are specified on the whole of it, or open if 
conditions are only specified on part of it. The boundary can of course include infinity; 
conditions at infinity are specified if the boundary is closed or unspecified if it is open. 
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Figure 9.56 

(a) A vibrating 

string fixed at its 
ends x = 0 and x =/; 
(b) the corresponding 
region and boundary 
conditions in the (x, f) 
plane; (c) wave 
moving forward 

with time. 
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Wave 
progresses 


Dirichlet condition 
Dirichlet condition 


Cauchy condition 


oO u, Ou/ot given x OW“ AB x 


(b) (c) 


The natural conditions for the wave equation are Cauchy conditions on an open 
boundary. The d’Alembert solution (9.15) in the (x, f) plane in Section 9.3.1 corresponds 
to u and du/dt given on the open boundary, t = 0. Physically these conditions correspond 
to a given displacement and velocity at time t = 0. However, the vibrations of a finite 
string, say a violin string, will be given by mixed conditions (Figure 9.56a). On the 
initial line 0 S x < /, t= 0 (Figure 9.56b) Cauchy conditions will hold, with both u 
and du/dt given. The ends of the string are held fixed, so we have Dirichlet conditions 
u=Oonx=0,t2=0andu=O0onx=/,t=0. 

Figure 9.56 is typical of the hyperbolic-type equations in the two variables x and ¢ 
that arise in wave propagation problems. For the second-order equation 


Au,, + 2Bu,, + Cu,, = 0 


the characteristics are defined by 


dy _ B+ \(B’ — AC) 
= ~ (9.83) 


For a hyperbolic equation, B? - AC > 0, so there are two characteristics, which for 
constant A, B and C are straight lines. Each of the characteristics carries one piece of 
information from the boundary into the solution region. This is illustrated in Figure 9.56(c), 
where the solution at P is completely determined from the information on AB. The pair 
of characteristics, TC and SC, then allows us to push the solution further into the region. 
It is clear from the d’Alembert solution that Cauchy data is required on the line t = 0 
but a single condition is required on the lines x = 0, J. 

There is no reason why the boundaries cannot be at infinity — an extremely long string 
can sensibly be modelled in this way. Care at such infinite boundaries must be taken, since 
the modelling of what happens there is not always obvious; certainly it requires thought. 

We have mentioned the commonest boundary conditions, but it is possible to 
conceive of others. However, such conditions do not always give a unique solution; a 
physical example will illustrate this point. 

Consider the problem in Example 9.1, which has the solution 


ae @ (2) 
u = uy sin| — | cos| — 
Es, iL 


Suppose that a photograph of the string is taken at the times t = L/2c and t= 3L/2c. Can 
the solution then be constructed from these two photographs? At the two times the 
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Figure 9.57 

(a) A heated bar with 
a temperature u = 0 at 
x = 0 and insulated, 
du/dx = 0, at x = L; 
(b) the corresponding 
region and boundary 
conditions in the (x, f) 
plane. (c) Solution 
can be computed at 
successive times. 


u=0[ J ux =0 
x=0 x=L 


(a) 


Solution 
progresses 
with time 


t 


Dirichlet condition 
Neumann condition 


Dirichlet condition 


oO u = f(x) given L x 


(b) (©) 


string has the same shape u = 0; that is, the string is in its resting position. One possible 
solution is therefore that the string has not moved. We know that a non-zero solution to 
Example 9.1 is possible, so we have two solutions to our problem, and we have lost 
uniqueness. Specifying the displacement at two successive times is not a sensible set of 
boundary conditions. Although we have stated an extreme case for clarity, the same 
problem is there for any 7, and non-unique solutions can occur if incorrect boundary 
conditions are imposed. 

The boundary conditions for the heat-conduction equation (9.7) for the one- 
dimensional case are given by specifying u or the normal derivative du/dn on a curve 
C in the (x, #) plane, that is Dirichlet or Neumann conditions respectively. Because there is 
only one time derivative in (9.7), we need only specify one function at t= 0 (say), rather 
than two as in the wave equation. In the simplest one-dimensional problem, at time ¢ = 0 
the temperature in a bar is given, u(x, 0) = f(x), and at the ends some temperature con- 
dition is satisfied for all time. Typical conditions might be (0, tf) = 0, so that the end x 
= 0 is kept at zero temperature, and Ou(L, t)/ox = 0, which implies no heat loss from the 
end x = L. The situation is illustrated in Figure 9.57. It is clear that, no matter what the 
starting temperature f(x) is, the solution must tend to wu = 0 as the final solution. 

In the case of a parabolic equation we have B? — AC = 0, so the characteristics 
in (9.83) coalesce. Imagine that there are two characteristics very close together. The 
information on the boundaries will propagate a long way, since the two lines will meet 
‘close to infinity’. We should therefore expect that information on the initial line would 
propagate forward in time, and because there is only one characteristic that one piece 
of information on the boundary curve would be sufficient. Figure 9.57(c) illustrates the 
situation, with the solution on CB being determined by a single boundary condition on 
each of CO, OA and AB. 

Again, as with the wave equation, there is no reason why the bar cannot be of infinite 
length, at least in a mathematical idealization, so that the initial curve C can include 
infinite parts. The conditions at infinity are usually quite clear and cause little difficulty. 

An interesting feature is that it is very difficult to integrate the heat-conduction equation 
backwards in time. Suppose we are given a temperature distribution at time t= T and seek 
the initial distribution at t= 0 that produces such a distribution of temperature at t= 7. If 
there is an exact solution then the problem can be solved, but it is unstable in the sense 
that small changes at t= T can lead to huge changes at t= 0. Consider, for instance, the 
solution to the heat conduction equation with k= 0.5 in the following two situations: 
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Given u= 0 on x =0 and I, and at r= 5 Given uw =0 on x= 0 and I, and at t=5 
u(x, 5) = sin(mx) e?™ u(x, 5) = sin(2mx) e!™ 

find u(x, 0). find u(x, 0). 

The solution is just one of the The solution is just one of the 

separated solutions in (9.40), namely separated solutions in (9.40), namely 
u(x, t) = sin(mx) e°"" u(x, t) = sin(2mx) e?"* 

Aiv=5 w= 2510" Ati=5 wo lAxie® 

Att=0 wu=sin(tx) Att=0 wu=sin(2Tx) 


The two conditions at ¢ = 5 differ by a very small amount (< 107) but, integrating 
backwards to t= 0, the two solutions are significantly different. Although this analysis 
is physically artificial it indicates why integrating backwards in time is unstable. This 
phenomenon, in particular, leads to almost insuperable difficulties when a numerical 
solution is sought, since errors are inherent in any numerical method. Such a situation 
applies, for instance, when a space capsule is required to have a specified temperature 
distribution on reaching its final orbit. The designer wants to know an initial temperature 
distribution that will achieve this end. 

The boundary conditions most relevant to the Laplace equation are Dirichlet or 
Neumann conditions. These specify respectively u or the normal derivative du/dn 
on a closed physical boundary. One condition around the whole boundary, which may 
include an infinite part, is sufficient for this equation. Typically, on a rectangular plate 
as shown in Figure 9.58, the temperature is maintained at 1 on CD, at 0 on AB and AD, 
and there is no heat loss from CB. 

However, it should be noted that for Neumann conditions, du/dn = f(s), on the whole 
boundary C, where s is a measure of length along the boundary, the function f(s) must 
satisfy an integral condition. Just consider the Laplace equation V7u = 0 in the region 
A with this boundary condition. In Section 3.4.5 Green’s theorem was written 


Pdx+QOdy= a0 oe dedy 
ox oy 
C 


A 


This can be re-written by putting P = —(du/dy), O = du/Ox to give 


aaa, +2lay = || Vudxdy = 0 
oy ox 
Cc A 


The right-hand side is put equal to zero since uw satisfies the Laplace equation. Thus 


= h (24 2) (dy —dxy = 2 
0= 4 (2 >) ea aa 


G ¢ 


and therefore 


pio ds=0 


(as 
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Figure 9.58 Typical 
boundary conditions 
for the Laplace 
equation in a 
rectangular plate. 


Figure 9.59 
Appropriateness of 
boundary conditions 
to the three classical 
partial differential 
equations (adapted 
from P. M. Morse and 
H. Feshbach, Methods 
of Theoretical Physics, 
Volume I. McGraw- 


Hill, New York, 1953). 


For the steady heat-conduction interpretation, du/dn is proportional to the heat entering 
through an element of the boundary. This result says that for a steady state to be 
achieved the net amount of heat entering the region must be zero. 

Figure 9.58 is typical of an elliptic equation where we have conditions on a closed 
boundary. In (9.83) we have the condition that B? — AC < 0, so the characteristics 
associated with the solution do not make sense in the real plane. There is no ‘time’ in 
elliptic problems; such problems are concerned with steady-state behaviour and not 
propagation with time. We are dealing with a fundamentally different situation from 
the hyperbolic and parabolic cases. The interpretation of the idea of characteristics is 
unclear physically, and does not prove to be a useful direction to explore, although 
advanced theoretical treatments do use the concept. 


Dirichlet condition 


Neumann condition 
{I 
Oo 


= 
2 
3 
< 
9 
3 
3 
= 
is) 
= 
o 


Dirichlet condition 


u=0 B x 


It is possible to solve the Laplace equation with other boundary conditions, for instance 
Cauchy conditions wu and du/Ox on the y axis. However, an example due to Hadamard 
(see Exercise 49) shows that the solution is unstable in the sense that small changes in 
the boundary conditions cause large changes in the solution. This type of problem is not 
well posed, and should not occur in a physical situation; however, mistakes are made 
and this type of behaviour should be carefully noted. 

Figure 9.59 gives in tabular form a summary of the appropriate boundary conditions 
for these problems. 


Data Boundary | V7u = wu, Vu =0 Vu = u, 
Hyperbolic Elliptic Parabolic 
Dirichlet Open Insufficient Insufficient Unique, stable 
or data data solution for t > 0 
Neumann 
Closed Not unique Unique, stable 
(to an arbitrary t 
constant in the Overspecined 
Neumann case) 
Cauchy Open Unique, Solution may 
stable exist, but is Overspecified 
unstable 
Closed Overspecified Overspecified Overspecified 
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9.8.3 Exercises 


Determine the type of each of the following 
partial differential equations, and reduce them 
to the standard form by change of axes: 


(a) w+ 2u,,+ u,,=0 

ani WE 
(b) ty + 2u, + 5u,, + 3u, +u=0 
(c) 3u,.— Su, — 2u,, = 0 
Find the general solution of the equation 
Exercise 68(c). 
Use the change of variable u=x+y,v=x-—y to 
transform the partial differential equation 

73 


(9.84) 


to 


Hence compute the general solution of equation 
(9.84) as 74 


S=(&— y)F(x + y) + Gx + y) 


where F and G are arbitrary functions. 


Establish the nature of the Tricomi equation 


VUyy + Uyy = 0 


in the regions (a) y > 0, (b) y= 0 and (c) y < 0. 
Use (9.83) to determine the characteristics of the 
equation where they are real. 


Verify that the function f= [Ax? + (BA’)] v1 — y”), 
with A and B constants, satisfies the partial 
differential equation 
eFt sa pyle 0 
Ox” oy 
In which regions is the equation elliptic, parabolic 
and hyperbolic? 


Determine the nature of the equation 
aq2® + 4p FY 4 2gF¥ 4 2% = 9 
op dpoq og og 
Show that if p = $(x°— y’) and g = $(x* + y’), 
the equation reduces to the Laplace equation 
in x and y. 


Show that the equation 


1d 20u 
x y mart 


oe 


= 0 
Ox oy 


is hyperbolic. Sketch the domain of 
dependence and range of influence from 
the characteristics. 


Ss et yedlal-\-) aleve] ye) fede Wave propagation under a 


moving load 


A wide range of practical problems can be studied under the general heading of moving 
loads. Cable cars that carry passengers, buckets that remove spoil to waste tips, and 
cable cranes are very obvious examples, while electric train pantographs on overhead 
wires are perhaps less obvious. Extending the problem to beams opens up a whole 
range of new problems, such as trains going over bridges, gantry cranes and the like. 
An excellent general discussion and wide range of applications is given by L. Fryba, 
Vibration of Solids and Structures under Moving Loads (Noordhoff, Groningen, 1973), 
and Initial Value Problems, Fourier Series, Overhead Wires, Partial Differential Equations 
of Applied Mathematics, Open University Mathematics Unit M321, 5, 6 and 7 (Milton 
Keynes, 1974) treat pantographs on overhead wires. 
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ae 
Pp 


Figure 9.60 Moving 
load across a taut wire. 


A straightforward linearized theory of a cable tightly stretched between fixed 
supports provides important information about the behaviour of such systems. Certainly, 
if such a theory could not be solved then more complicated problems involving 
large deformations, slack cables or beams would be beyond reach. The basic assump- 
tions are 


(a) the deflections from the horizontal are small compared with the length of the cable; 
(b) deflections due to the weight of the cable itself are neglected; 


(c) the horizontal tension in the cable is so large compared with the perturbations 
caused by the load that it may be regarded as constant. 


Figure 9.60 shows the situation under study and the coordinate system used. 

Because the problem is one of small deflections, the basic equation is the wave 
equation with a forcing term from the moving load: 
geo 


Cc a oF = p(x, t) 


Since the two ends are fixed, we have 
2(0, )=2(,)=0 (t=0) (9.85) 


A trolley is assumed to start at x = 0, with the cable initially at rest; that is, 
z(x, 0) = $ 2, 0=0 (0SxS/) (9.86) 


It remains to specify the forcing function p(x, ft) due to the moving load. We use the 
simplest assumption of the delta function and step function, as defined in Section 5.5, 
namely 
pee + Jz 7 Poe = ) HU — x) (9.87) 
ox” ot v 


The delta function models the impulse of the trolley at time ¢ at distance x, while the 
step function switches off the forcing function when the trolley reaches the end x = /. 
There are several ways of solving this equation, but here the Laplace transform 


method will be used. Taking the transform of (9.87) using (9.30) and (9.32) together 
with the initial condition (9.86), we obtain the ordinary differential equation 


aes” fs vA = P oe" A(L _ x) 


Since we have no interest in the case x > /, the final term can be omitted, since it is just 
lifx <J/and0ifx > /. It is now straightforward to solve this equation as 


—sx/v 
ZHAO + Bete Ore 
s(l-—c/v’) 

Before evaluating A and B, it is clear that the speed v = c causes problems, since the 
third term is then infinite, and the solution is not valid for this case. The solution is 
going to depend on whether the trolley speed is subcritical v < c or supercritical v > c. 

Equation (9.85) gives Z(0, s) = Z(/, s) = 0 for the boundary conditions, so that A and 

B can now be evaluated from 
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P 


0=A+B+———_— 
s(1-¢'/v’) 


P el 


0 =Ae™ +Bew 4 
s(1-c’/v’) 


Some straightforward algebra gives A and B, and hence Z, as 


z=- P — = _ -wesinh(sx/e) , sinh[s(x = ball 
s(l-c/v) sinh(s//c) sinh(s//c) 


It is easy to check that the two boundary conditions at x = 0 and x = / are satisfied. As 
with all transform solutions, the main question is whether the inversion can be per- 
formed. Fortunately the three terms can be found in tables of transforms, to give 


seer are 
=H D(-D+ BESP nman()- 2) 


1), USED 5 fame) «| ance 
+{( \e+ =? ~ a [si] |} (9.88) 


Cc 


n=1 


The three terms can be identified immediately. The first is the displacement caused by 
the trolley moving with speed v and hitting the value x after a time t = x/v; the second 
term only appears for tf > //v, and gives the reflected wave from x = /; while the third 
term is the wave caused by the trolley disturbance propagating in the cable with wave 
speed c. 

To look a little more closely at the solution (9.88), we shall consider the case x = ; 1. 
Thus the motion of the midpoint will be considered as a function of time. Plotting such 
waves is easier in non-dimensional form, so we first rewrite (9.88) in terms of 


2 
p=2(1 -§), eo G28 
l v 
so that D is the non-dimensional displacement, T is the non-dimensional time, with T= 1 
corresponding to the time for the wave to propagate the length of the cable, and A is the 
ratio of wave speed to trolley speed. The second step is then to take x = bl to give 


D=(t- 4AH(t- 3d) 


—HA(t- afc A) - 2 fsin x(t - A) — isin 3n(t — A) 
T 


+ #sin 5x(t — A) If T+ 2 (sinnt - tsin3nt + + sin 507...) 
> Tt 


In Figure 9.61 the supercritical case, A = 0.3, is displayed. It may be noted that the 
three terms ‘switch on’ at times T = 0.15, T= 0.8 and tT = 0.5 respectively, corresponding 
physically to the trolley hitting, the reflected wave arriving and the initial wave arriving. 
The motion is subsequently periodic, as indicated in the figure. 
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Figure 9.61 Solution 
of the moving-load 
problem for x = }/ 
and A = 0.3; the 
supercritical case. 


Similarly, the subcritical case 2 = 3 is shown in Figure 9.62. Here the switches are 
at T= 1.5, 2.5 and 0.5 respectively for the three terms, with the same interpretation as 
above. Because of the very odd choice of parameter A, only one pulse is seen at the 
centre point, with the terms subsequently cancelling. 


Figure 9.62 Solution 
of the moving-load 
problem for x = }/ and 
A = 3; the subcritical 
case. 


While the model illustrates many of the obvious properties of wave propagation, it 
clearly has its limitations. The discontinuous behaviour in the gradient of the displacement 
looks unrealistic, and the absence of damping means that oscillations once started con- 
tinue for ever. It is clear that more subtle modelling of the phenomenon is required to 
make the solutions realistic, but the general behaviour of the solution would still be followed. 


9.10 Engineering application: Me)feYere Rita rirere (sy 


A problem of considerable interest is how to deal with the flow of a fluid through a tube 
with distensible walls and hence variable cross-section. An obvious application is to 
the flow of blood in a blood vessel. The full Navier-Stokes equations for viscous flow 
are difficult to solve and the distensible wall, where boundary conditions are not clear, 
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Figure 9.63 

An element of the 
flexible tube in the 
blood-flow problem. 
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makes for an impossible problem. An alternative, simpler and more heuristic approach 
is possible and some useful solutions can be deduced. The work is based on a paper by 
A. Singer (Bulletin of Mathematical Biophysics 31 (1969) 453-70), where more details 
can be found about the practical application to blood flows. 

The assumptions required to set up the model are as follows: 


(1) _ the flow is one-dimensional; 

(2) the flow is incompressible and laminar; 

(3) _ the flow is slow, so that all quadratic terms can be neglected; 

(4) _ the resistance to flow is assumed to be proportional to the velocity; 

(5) there is a leakage through the walls that is proportional to the pressure; 
(6) the cross-sectional area S is a function of the pressure only. 


We take the situation illustrated in Figure 9.63, and we denote the pressure by p, the 
velocity by v, the time by ¢ and the axial distance along the tube by x. The first equation 
that we derive is a continuity equation, which states that in a time Af the fluid that comes 
into the element must leave the element: 


(Star — SAX = —(US)raxAt + (vS),At — gpSAxAt 


volume volume volume out of volume into leakage 
after before right-hand end left-hand end 


The proportionality constant g is the leakage per unit volume of tube per unit time. The 
equation can be rewritten as 


Sisar = Sy, US)xzax = US) x 
2th Pp, UO eke LO a ee 
At Ax = 


or 


os , a 

= + =(vS) + gpS =0 9.89 
yo ee ee (9.89) 
A second equation is required to evaluate v, and this comes from Newton’s law that the 
force is proportional to the rate of change of momentum. The force in the x direction 


acting on the element in Figure 9.63 is 


force= (pS), -— (pS)yaa, — vrSAx 


pressure force pressure force resistance 
on left-hand on right-hand 
end end 


where + is the resistance per unit length per unit cross-section per unit time, and is the 
proportionality constant in assumption (4). The change in the momentum in time Af is 
more difficult to compute because of the convection due to the moving fluid. However, 
these effects only involve second-order terms, and hence can be omitted by assumption 
(3). The calculation is straightforward under this assumption, so that 


x x+Ax 
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AM = change in momentum = p(v,.,,)SAx — pl(v)SAx 


momentum before momentum after 
where p is the density of the fluid. Thus 


aM _ av 
Ot =e. 


Putting the force equal to the rate of change of momentum, we obtain, on taking the 
limit as Ax > 0, 


ps2 = APD _ ys (9.90) 
X 


Now assumption (6) gives S = S(p), so that 


1S _ 148 dp 
Sox  Sdp ox 
1S _ 14S dp 
Sot Sdp ot 


We define c = (1/S') dS/dp as the distensibility of the tube, that is the change in S per 
unit area per unit change in p. Equations (9.89) and (9.90) become 


CP, +0, + cp,v + gp = 9 
pu, + Pp, + cp,.p +rv=0 


and since the terms up, and pp, can be neglected by assumption (3), we arrive at our 
final equations 


+v,+ gp =90 
dale (9.91) 


pu, +p, +trv=0 


These are the linearized flow equations, and are identical with the transmission line 
equations describing the flow of electricity down a long, leaky wire such as a trans- 
atlantic cable (see Exercise 10). 

We can now look at special cases that will prove to be very informative about the 
various terms in the equation. 


Case (i): c = constant, r= ¢ =0 


This case corresponds to constant distensibility, which in turn gives S = Ae”, since S$ 
must satisfy c = (1/S) dS/dp. Thus we have made a specific assumption about how S 
depends on p. The r = g = 0 implies the absence of resistance and leakage. Eliminating 
p between the two equations in (9.91) gives 


Dx. = (CPW, 


which is just the wave equation. We know that any pulse will propagate perfectly with 
a velocity u = 1/\(cp). The assumption in the problem is that the tube is one-dimensional 
and has no branches. Clearly a heart pulse will propagate to the nearest branch, but 
there will then be reflection and a complicated behaviour near the branch. In long 
arteries like the femoral artery the theory can be checked for its validity. 
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Figure 9.64 
Development of 

the solution to the 
blood-flow problem 
from a delta function 
for successive times ¢,, 
t, and £5. 
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Case (ii): S = constant 


Here we are considering a rigid tube where the cross-sectional area does not vary, and 
hence c = 0. Eliminating p between the two equations in (9.91) gives 


pv,- 1, +rv=0 
& 


Substituting v = Ve’, we have 


which is just the diffusion equation. The solution for this rigid-tube case is therefore 
a damped, diffusion solution. Typically, if we start with a delta-function pulse at the 
origin then it can be checked that the solution is 


p 
e —pgx?/4t 
v=A——e ® 


where A is a constant. This solution is plotted in Figure 9.64, and shows the rapid damp- 
ing. Such a pulse would be most unlikely to propagate far enough for blood to reach the 
whole of the system. 


The two cases considered are extremes, but, just from the analysis performed, some 
conclusions can be drawn. If there is no distensibility then pulses will not propagate but 
will just diffuse through the system. We conclude that to move blood through the sys- 
tem with a series of pulses is not possible with rigid blood vessels, and we need flexible 
walls. Certainly for older people with hardening of the arteries, a major problem is to 
pump blood round the whole system, and this fact is confirmed by the mathematics. 

The actual situation is somewhere between the two cases cited, but there are no 
simple solutions for such cases except for the ‘balanced line’ case when cr = gp (see 
Exercise 10(c) and Review exercise 20). Singer solves the equations numerically for 
data appropriate to a dog aorta, and compares his results with experiment. Although the 
agreement is good, there are problems, since there appears to be a residual pressure 
after each pulse. The overall pressure would therefore build up to levels that are clearly 
not acceptable. 
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9.11 Review exercises (1-21) 


1 =A uniform string is stretched along the x axis and its 
ends fixed at the points x = 0 and x = a. The string 
at the point x =b (0 < b <a) is drawn aside through 
a small displacement € perpendicular to the x axis, 
and released from rest at time ¢ = 0. By solving the 
one-dimensional wave equation, show that at any 
subsequent time ¢ the transverse displacement y is 
given by 


where c is the transverse wave velocity in the string. 


2 _ The function @(x, A) satisfies the wave equation 


7.2 (¢>0,0<x</) 
Xx 


and the conditions 
(x, 0) =x? (O<x</) 
2 0)=0 O<x<I) 
ot 
9(0,)=0 (¢>0) 
29 «1 2/10) 
ox 
Show that the Laplace transform of the 


solution is 


y w 8 cosh s/ 


Using tables of Laplace transforms, deduce that the 
solution of the wave equation is 


Neale a (-1)" - 0s 2n+1)(x-l)n 
ane iy 21 


Xx COS 


ee a 
21 


3 The damped vibrations of a stretched string are 
governed by the equation 


= 2 (9.92) 


where (x, f) is the transverse deflection, ¢ is the 
time, x is the position coordinate along the string, 
and c and T are positive constants. A taut elastic 
string, 0 < x < /, is fixed at its end points so that 


(0) = y(/) = 0. Show that separation of variable 


solutions of (9.92) satisfying these boundary 
conditions are of the form 


y,(x, #) = 7,(0)sin{ (n=1,2,...) 


where 


d° dene 
= +——— + ——— 


= (0) 
ec df ctdt iP 


1 
B 


Show that if the parameters c, tT and / are such 
that 2mct > /, the solutions for 7, are all of the 
form 


T(t) = '"(a, cos @,t + b, sin @,f) 


where 


nic iF a 
ee (1 =) 
I 40 n'c 


and a, and 5, are constants. 

Hence find the general solution of (9.92) 
satisfying the given boundary conditions. 

Given the initial conditions y(x, 0) = 4 sin(3nx//) 
and (Oy/0t),-» = 0, find (x, ft). 


A thin uniform beam OA of length / is clamped 
horizontally at both ends. For small transverse 
vibrations of the beam the displacement u(x, ft) 
at time ¢ at a distance x from O satisfies the 
equation 


Oe wr ahi 
where a is a constant. The restriction that the 


beam is clamped horizontally gives the boundary 
conditions 


du 
‘Ox 


u=0, =0 (@=0,/) 


Show that for periodic solutions of the type 


u(x, t) = V(x) sin(@t + €) 
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where wand € are constants, to exist, V must satisfy 
an equation of the form 


4 
dV _ ov (9.93) 
dx 
where of = (@/a)*, and the boundary conditions 
M0) = V0) = Vl) = Vl) = 0 


Verify that 


V=Acoshax + Bcos ax + Csinhax 
+ Dsin ax 


where A, B, C and D are constants, satisfies (9.93), 
and show that this function satisfies the boundary 
conditions provided that 


B=-A, D=-C 
and @ is a root of 


cos alcosha/=1 


In a uniform bar of length / the temperature O(x, f) 
at a distance x from one end satisfies the 
equation 


Po _ 220 
ax? ot 


where a is a constant. The end x = / is kept at 
zero temperature and the other end x = 0 is 
perfectly insulated, so that 


a(,)=0, 20,)=0 @>0) 
Using the method of separation of variables, 


show that if initially the temperature in the bar is 
O(x, 0) = f(x) then subsequently the temperature is 


A(x, ) =H Anat cos| nes 


n=0 


22 
x op Ou | 
4a l 


where 


if 
Aayst = 2 f(x) cos aE dx 
0 


Given @(x, 0) = @,(/ — x), where ) is a constant, 
determine the subsequent temperature in the bar. 
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Prove that if z= x/,t and @(x, t) = f(z) satisfies the 
heat-conduction equation 


«2b 26 (9.94) 
OxemnOl 


then f(z) must be of the form 


fe)=Aet(37) +B 


where A and B are constants and the error 
function is defined as 


iS 
erf(§) = ral e" du 
yu 


0 


A heat-conducting solid occupies the 
semi-infinite region x > 0. At time t= 0 the 
temperature everywhere in the solid has the 
value 7). The temperature at the surface, x = 0, 
is suddenly raised, at t = 0, to the constant value 
T) + ) and is then maintained at this temperature. 
Assuming that the temperature field in the solid 
has the form 


T= (x, 1) + Ty 


where @ satisfies (9.94) in x > 0, find the solution 
of this problem. 


Use the explicit method and the Crank—Nicolson 
formula to solve the heat-conduction equation 


Jo _ ao 
ox at 


given that @ satisfies the conditions 


g=1 OS<x<1,t=0) 


Compute (x, #) at x = 0, 0.2, 0.4, 0.6, 0.8, 1 when 
t = 0.004 and t= 0.008. 


An infinitely long bar of square cross-section has 
faces x = 0, x =a, y= 0, y =a. The bar is made of 
heat-conducting material, and under steady-state 
conditions the temperature T satisfies the Laplace 
equation 


ax ay 


0 


840 


10 


ial 
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All the faces except y = 0 are kept at zero 
temperature, while the temperature in the face 
y = 0 is given by 7(x, 0) = x(a — x). Show that 
the temperature distribution in the bar is 


8a" x sin[(2r + 1)nx/a] sinh[(2r+ 1)n(a- y)/a] 
eam (2r+1) sinh(2r+ 1) 


(Harder) The function @ satisfies the Poisson 
equation 


inside the area bounded by the parabola y” = x and 
the line x = 2. The function @ is given at all points 
on the boundary as @= 1. By using a square grid of 
side } and making full use of symmetry, formulate 
a set of finite-difference equations for the unknown 
values @, and solve. 


A semi-infinite region of incompressible fluid of 
density p and viscosity is bounded by a plane wall 
in the plane z = 0 and extends throughout the region 
z = 0. The wall executes oscillations in its own 
plane so that its velocity at time tis U cos wt. No 
pressure gradients or body forces are operative. It 
can be shown that the velocity of the liquid satisfies 
the equation 


Ou _ Ju 
gu yok 
ot oz 


where v = 1/p. Establish that an appropriate 
solution of the equation is 


u= Ue cos(@t — az) 


where & = |(@/2v). 


Determine the value of the constant k so that 
U = ther 
satisfies the partial differential equation 


12 (p20)_au 
r or or Ot 


Sketch the solution for successive values of t. 


The function z(x, y) satisfies 


Oz oz _ 
Fp eis 


13 


14 


tS) 


with the boundary conditions 
z=2x wheny=-x (x>0) 


Find the unique solution for z and the region in 
which this solution holds. Check the solution 
using MAPLE. 


The function @(x, y) satisfies the Laplace equation 
ao 4. ao = 0 
ax oy’ 


in the region 0 < x < m1, 0 < y, and also the 
boundary conditions 


¢7>0 asyro 


(0, v) = 9(T, y) = 0 


Show that an appropriate separation of variables 
solution is 


o= >) c, sin(nx) e”” 


n=1 


Show that if further 
(x, 0) = x(t — x) 
then c;,, = 0 while the odd coefficients are given by 


eee 
m(2m+1)° 


Come = 


The boundary-value problem associated with the 
torsion of a prism of rectangular cross-section 
-—a <x <a,-b <y < b entails the solution of 
i ‘a 

ox =f Tx =-2 

ox oy 
subject to y = 0 on the boundary. Show that the 
differential equation and the boundary conditions 
on x = +a are satisfied by a solution of the form 


= ip mse DY Arne cosh 2n+ Iny 
2a 


n=0 


2a 


Se cos +1 x 


From the condition v = 0 on the boundaries y = +b, 
evaluate the coefficients A,,,,. 


When 0 < x < 1 and¢ > 0 the function u(x, £) 
satisfies the wave equation 
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16 


Ly 


18 


ax or 
and is also subject to the following boundary 
conditions: 


(a) u(0, t)=u(1, 4) =0 forallt > 0 


(b) ae 20 O21) 


(c) u(x, 0)=1-x (0<x<1) 


Use the separation method to find the solution for 
u(x, t) that is valid for0 <x < 1 andt> 0. 


The excess porewater pressure u(z, ¢) in an infinite 
layer of clay satisfies the diffusion equation 


du _ Fu (= 030 2h) 


ot oz 


where f is the time in minutes, z is the vertical 
height in metres from the base of the clay layer 
and c is the coefficient of consolidation. There is 
complete drainage at the top and bottom of the clay 
layer, which is of thickness h. The distribution 

of excess porewater pressure u(z, ¢) is A at t= 0 
where 4 is a constant. Show that 


Wee ie tA ‘ sin[(2n + 1)nz/h] em enstyane 


2n+1 
n=0 


By seeking a separated solution of the form 


@ = X(x)7(0), find a solution to the telegraph 
equation 


74.1 (28, 20) 
ax c\odf at 


satisfying the conditions 


(a) @=Acos px for all values of x and for t= 0 
for the case when c*p* > + K?; 
(b) ¢=A and Og/ot = -}AK for x = 0 and t= 0. 


For the two-dimensional flow of an incompressible 
fluid the continuity equation may be expressed as 


OVg 
00 


where r and @ are polar coordinates in a 

plane parallel to the flow, and v, and v, are the 
respective velocity components. Show that a stream 
function y such that 


2 (u,)+B =0 
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ay 
“~ Pa 
ae _@ 
4 or 
satisfy the continuity equation. 
Take 


2 
w= Ur sin 0—- ee sin 0 
r 
and interpret the solution physically. 


(An extended problem) Section 9.9 looked at 
wave propagation caused by moving loads on 
cables. For loads on beams a similar analysis 
models such problems as trains going over 
bridges or loads moving on gantry cranes. 
Use a similar analysis for the beam equation 


Pre Ju 


ae gs oe 
x" or 


(An extended problem) In the blood-flow model 

in Section 9.10 consider the following cases: 

(a) S=constant, g = 0 for a pulsating flow 
v=veil atx=0 for allt 

(b) S=constant, r = 0 for a pulsating flow 
v=vei atx=0 forall t 


(c) the balanced-line case when gp = rc. Show that 
v=e*'"U gives 
cOL Mca) 
Ot cp ax 


Solve the equation and interpret your solution. 


(An extended problem) Fluid flows steadily in the 
two-dimensional channel shown in Figure 9.65. 
The temperature 0 = 0(x, t) depends only on the 
distance x along the channel and the time ¢. The 


x O x+6x 


Figure 9.65 An element of the channel in Review 
exercise 21. 
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fluid flows at a constant rate so that an amount L 
crosses any given section in unit time. The specific 
heat of the fluid is a constant c, and the heat Hin a 
length 6x with cross-section S is therefore 


H=c(S6x)@ 


Heat is transferred through the walls of the 
channel, AB and DC, at a rate proportional to the 
temperature difference between the inside and 
outside. Heat conduction in the x direction is 
neglected. Show that the heat balance in the 
element ABCD leads to the equation 
ose! = ie2” EG" — 0) Ee — 0) 
ot Ox 

This type of analysis can now be applied to the 
long heat exchanger illustrated in Figure 9.66. The 
configuration is considered to be two-dimensional 
and symmetric with respect to the x axis; in the 
inner region the flow is to the right, while in the 
outer regions it is to the left. The regions are 
separated by metal walls in which similar 
assumptions to the above are made, except 
of course there is no fluid flow. 

Set up the equations of the system in the form 


00 00 

any yi =-L,c, oe + 2k,(8, — 0;) 
00 

oS > = —k,(0; — @) + k(8; — @) 
00 00. 

es >> = Les + k,(0, — 03) 


O3(x, 0D) ~<«—l, 
VMAs 


A(x, 0) 
VMAS itll 


O3(x, ft) bs 


Figure 9.66 Heat-exchanger configuration in 
Review exercise 21. 


where the assumption is made that there is no 
heat flow through the outside lagged walls. Solve 
the steady-state equations and fit the arbitrary 
constants to the conditions that at the inlet 
(x = 0) the fluid enters the inner region at a given 
temperature 0, = 7,, while at the outlet (x — -) 
the fluid in the outer regions enters at a given 
temperature 0; = 7. Find flow rates that ensure 
that this situation is possible, and discuss the 
implications of any results obtained. 

Discuss the assumptions made in setting 
up this problem, the limitations imposed by the 
assumptions, possible applications of this type 
of analysis, and extensions of the work, for 
example a time-dependent solution. 
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10.1 


Example 10.1 


Solution 


Introduction 


The need to get the ‘best’ out of a system is a very strong motivation in much of 
engineering. A typical problem may be to obtain the maximum amount of product or to 
minimize the cost of a process or to find a configuration that gives maximum strength. 
Sometimes what is ‘best’ is easy to define, but frequently the problem is not so clear 
cut, and a lot of thought is required to reach an appropriate function to optimize. In most 
cases there are very severe and natural constraints operating: the problem may be one 
of maximizing the amount of product, subject to the supply of materials; or it may be 
minimizing the cost of production, with constraints due to safety standards. Indeed, 
much of modern optimization is concerned with constraints and how to deal with them. 

We have seen in Chapter 9 of Modern Engineering Mathematics how to obtain the 
maximum and minimum of a function of many variables. However, the methods described 
there founder very quickly because it becomes impossible to solve the resulting 
equations analytically. A simple one-dimensional example soon shows that a numerical 
solution is required. 


Find the positive x value that maximizes the function 


_ tanh x 
1+x 


Equating the derivative to zero gives 


dy _ 0= (1 +x) sech’x - tanh x 
dx (1+x) 


so that we need to solve 
1+x= sinh 2x 


which has no simple positive solutions that can be obtained analytically. 


To solve such problems, a set of numerical algorithms was developed during the 
1960s as fast computers became available to perform the large amounts of arithmetic 
required. These algorithms will be described in Section 10.4. Perhaps the main stimulus 
for this development came from the space industries, where small percentage savings, 
achieved by doing some mathematics, could save vast amounts of money. The ideas 
were quickly taken up by ‘expensive’ areas of engineering, such as the chemical and 
steel industries and aircraft production. 

The idea of dealing with constraints is not new: Lagrange developed the theory of 
equality-constrained optimization around the 1800s. However, it was not until the 1940s 
that inequality constraints were studied with any seriousness. The use of Lagrange 
multipliers for equality constraints was also introduced in Chapter 9 of Modern Engin- 
eering Mathematics, and will be looked at again in more detail in Section 10.3 below. 
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Example 10.2 


Figure 10.1 Waxed 
cardboard milk 
container opened up, 
with measurements in 
millimetres and with a 
5mm overlap. 


Solution 


10.1 INTRODUCTION 845 


The only work on inequality constraints will be for linear programming problems in 
Section 10.2. Where inequality constraints are nonlinear, the problems become very 
difficult, and are the province of specialist books on optimization. Linear programming, 
however, is much more straightforward, and the basic simplex algorithm has been spec- 
tacularly successful — so successful in fact that many workers try to force their problems 
to be linear when they are clearly not. The computer scientist’s maxim GIGO (‘garbage 
in, garbage out’) is very applicable to people who try to fit the problem to the mathem- 
atics rather than the mathematics to the problem! 

Before considering detailed methods of solution of optimization problems, we 
shall look at a few examples. Let us first revisit an extended form of the milk 
carton problem considered in Example 8.34 (and illustrated in Figure 8.38) of Modern 
Engineering Mathematics. 


A milk carton is designed from a sheet of waxed cardboard as illustrated in Figure 10.1, 
where a 5mm overlap has been allowed. 
It is to contain 2 pints of milk, and we require the minimum surface area for the carton. 


5 mm 
i Ke 
5 mm 
T 1p 
Fresh Milk 
2 pints 
1.136 litres 
v 2 
5mm 
TK 3K 3K >K 3K >I 
iw b w b iw 


The only difference between this example and Example 8.34 of Modern Engineering 
Mathematics is that we no longer insist on a square cross-section. The total area in 
square millimetres is 


A=(2b+2w+ 5)(h+6+ 10) 
and the volume of the two-pint container is 
volume = hbw = 1 136000 mm?’ 


We first note that a constraint, the given volume, occurs naturally in the problem. Because 
of its simplicity, we can eliminate w from the constraint to give 


2 272 000 


A=(h+b+10){ hb 


+2645) 


Following the standard minimization procedure and equating partial derivatives to 
zero gives 
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Figure 10.2 
The moonshot 
problem. 


QA _ 2272000 2 272000 _ 


+2b+5-(h+b+10) ; 0 
ah hb ib 
dA _ 2212000, 2b-+5~(h+b+10)( 2222000, 2) - 9 
db hb hb 


We therefore have two highly nonlinear equations in the two unknowns / and b, which 
cannot be solved without resorting to numerical techniques. We shall return to this problem 
later in Examples 10.11 and 10.17 to see how a practical solution can be obtained. 


Most practical optimization problems come from very expensive projects where 
savings of a few per cent can be very significant. Laying natural gas or water pipe 
networks are typical examples. Without considering the expense of installing com- 
pressors, the problem is to minimize the capital cost. This cost is directly related to the 
weight of the pipe, subject to constraints imposed by pressure-drop limitations, which in 
turn depend on the pipe diameter in a nonlinear way. Adding the compressors imposes 
further costs and constraints. 

Heat exchangers provide an example of a system where we try to remove heat. We 
design the flow rates, the pipe sizes and pipe spacing to maximize the heat transferred. 
A related heating problem might be the design of an industrial furnace. It is required 
that the energy consumption be minimized subject to constraints on the heat flow and 
the maintenance of various temperatures. 

A final example, the moonshot problem, illustrates a large-scale, very complicated 
problem that stimulated much of the recent developments in optimization (see Figure 10.2). 
Which path from a point on the Earth to a point on the Moon should be chosen to 
minimize the weight of fuel carried by a rocket? The complicated relation between the 
weight of fuel, the mechanical equations of the rocket and the path must be established 
before it is possible to proceed to obtain the optimum. The numerous constraints on the 
strengths of materials, the maximum tolerable acceleration etc. add to the difficulty 
of the problem. 

In the problems discussed we have assumed that an optimum exists at a point, and 
we have asked for the mathematical conditions that must hold. The other way round is 
much more difficult. Given that the appropriate conditions hold, does an optimum exist, 
and if so what type of optimum is it? For many simple finite-dimensional problems 


Moon 
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10.2.1 


Example 10.3 


Solution 


10.2 LINEAR PROGRAMMING 847 


these conditions are known, but may not be very simple to apply. To serve as a reminder, 
the condition f’(0) = 0 is a necessary condition for a maximum to exist for the differ- 
entiable function f(x) at x = 0. It is not sufficient, however, as can be seen from the three 
functions f,(x) = x7, f(x) = »° and f,(x) = —x’, which have respectively a minimum, a 
point of inflection and a maximum at the origin. In many dimensions the difficulties are 
similar, but much more complicated. 


Linear programming 


Introduction 


In Section 10.1 it was indicated that constraints are very important in most applications. 
When all functions are linear, there is an extremely efficient algorithm, developed by Danzig 
in the 1940s, which will be described for the linear programming (LP) problem. 

We shall start by posing a particular problem and looking at a simple graphical 
solution. 


A manufacturing company makes two circuit boards R1 and R2, constructed as follows: 


R1 comprises 3 resistors, 1 capacitor, 2 transistors and 2 inductances; 
R2 comprises 4 resistors, 2 capacitors and 3 transistors. 


The available stocks for a day’s production are 2400 resistors, 900 capacitors, 1600 
transistors and 1200 inductances. It is required to calculate how many R1 and how many 
R2 the company should produce daily in order to maximize its overall profits, knowing 
that it can make a profit on an R1 circuit board of 5p and on an R2 circuit board of 9p. 


If the company produces daily x of type R1 and y of type R2 then its stock limitations give 


3x + 4y = 2400 (10.1a) 

x+2y <= 900 (10.1b) 
2x + 3y < 1600 (10.1c) 
2x = 1200 (10.1d) 
x20, y20 


and it makes a profit z given by 
z=5x+4+9y (10.2) 


These inequalities are plotted on a diagram as in Figure 10.3(a). The shaded region defines 
the area for which all the inequalities are satisfied, and is called the feasible region. The 
lines of constant profit z = constant, defined by (10.2), are plotted as ‘dashed’ lines in 
Figure 10.3(b). It is clear from the geometry that the largest possible value of z that 
intersects the feasible region is at S with x = 500, y = 200, and this gives the optimal 
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600 -- 


400 600 800 x 


(a) (b) 


Figure 10.3 (a) Feasible region for the circuit board manufacture problem. (b) Lines of constant z show that S (500, 200) 


gives the optimum. 


Figure 10.4 Table 
stock usage. 


of 


solution. At this point we can analyse the usage of the stocks as in Figure 10.4 and note 
that a profit of £43 has been made. 


Available Used Left over 
Resistors 2400 2300 100 
Capacitors 900 900 0 
Transistors 1600 1600 0 
Inductances 1200 1000 200 


Example 10.3 has encapsulated much of the LP method, and we shall try to extract 
the maximum amount of information from this example. The graphical method will 
only work if the problem has two variables, so we need to consider how to translate the 
geometry into an algebraic form that will work with any number of variables. Although 
we shall concentrate in this chapter on small problems in order to illustrate the methods, 
in practical problems there can be hundreds of variables and constraints. Large prob- 
lems bring further difficulties that will not be considered here; for instance, how a large 
amount of information can be input into a computer accurately or how large data sets 
are handled in the computer. In the MATLAB implementation of LP, there is a specific 
option to deal with ‘LargeScale’ problems. 

From Figure 10.3 it can be seen that the solutions must be at a ‘corner’ of the feasible 
region, other than in the exceptional case when the profit line z = constant is parallel to 
one of the constraints. This follows through into many-dimensional problems, so that it is 
only necessary to inspect the corners of the feasible region. The simplex method, described 
in Section 10.2.3, uses this fact and selects a starting corner, chooses the neighbouring 
corner that increases z the most, and then repeats the process until no improvement is 
possible. The method writes the equations into a standard form; it then automates the 
choice of corner and finally reprocesses the equations back to the standard form again. 

Once a solution has been obtained, it may be observed from Figures 10.3 and 10.4 
that the binding constraints (b) and (c) intersect at S and are satisfied identically, so 
that all the stocks are used, while the non-binding constraints (a) and (d) leave some 
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stock unused. It can also be seen from Figure 10.3 that the constraint (a) is redundant 
since it does not intersect the feasible region. These might appear obvious comments, 
but they prove to be useful and relevant observations when a sensitivity analysis is 
performed. Such an analysis asks whether or not the solution changes as the stocks vary 
or the costs vary, or the coefficients are changed. In practice, parameters vary over a 
period, and we wish to know whether a new calculation must be performed or whether 
the solution that we have already obtained can be used. 
There are a whole series of special cases that we must consider: 


(a) Does a feasible region exist? If the system is modelled correctly, a feasible region 
always exists for a sensible problem; but if an error is made then it is easy to eliminate 
the region completely. For instance, in (10.1d) putting —1200 instead of 1200 would 
give an empty region, and this must be detected by any program. 


(b) Is the solution degenerate? If the profit lines z = constant are parallel to any one of 
the final binding constraints then any feasible part of the constraint gives a solution. 


(c) Can we get unbounded regions and solutions? Again it is very easy to construct 
problems where the regions are unbounded and finite solutions may or may not 
exist. Just maximizing (10.2) subject only to the single constraint (10.1d) provides 
an unbounded region. Interpreting this situation, the only constraint that we have 
is on the inductances. If this were true, we could make infinitely many circuits of 
type R2 and make an infinite profit! 


Simplex algorithm: an example 


We now need to convert the ideas of Section 10.2.1 into a useful algebraic algorithm. 
There is a whole array of technical terms that are used in LP, and they will be intro- 
duced as we reconsider Example 10.3 to develop the solution method. The first step is 
to introduce slack variables r, s, ¢ and u into (10.1) to make the inequality constraints 
into equality constraints. 

If we are given x and y in the feasible region, the variables r, s, t and u provide a 
measure of how much ‘slack’ is available before all the corresponding resource is used 
up, so 


3x+4y+r = 2400 (10.3a) 

x+2y +5 = 900 (10.3b) 
2x + 3y +t  =1600 (10.3c) 
2x +u= 1200 (10.3d) 


where x, y, 7, Ss, ¢ and u are now all greater than or equal to zero. We now have more 
variables than equations, and this enables us to construct a feasible basic solution by 
inspection: 


x=y=0; r=2400, s=900, += 1600, w= 1200 
ee ——_Y | 
non-basic basic variables 
variables 


with 4 basic variables (the same number as constraints, which are non-zero) and 2 non- 
basic variables (the remainder of the variables, which are zero). 
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The algebraic equivalent of moving to a neighbouring corner is to increase one of 
the non-basic variables from zero to its largest possible value. From the profit function 
given in (10.2), we have 

z=5x+9y 


Currently z has the value zero, and it seems sensible to change y, since the coefficient 
of y is larger; this will increase z the most. So keep x = 0 in (10.3) and increase y to its 
maximum value in each case: either 

(a) change y to 600 and reduce r to zero, or 

(b) change y to 450 and reduce s to zero, or 

(c) change y to 533 and reduce f to zero, or 

(d) note that there is no effect on changing y. 

Choose option (b), since increasing y above 450 will make s negative, which would then 
violate the condition that all variables must be positive. Interchange s and y between the 


set of basic and non-basic variables and rewrite in the same form as (10.3). This is 
achieved by solving for y from (10.3b), y = 450 — $x — }s, and substituting to give 


x-2s+r = 600 (10.4a) 
ae + ae +y = 450 (10.4b) 
5x — 38 +t = 250 (10.4c) 
2x +u= 1200 (10.4d) 
and, from (10.2), 
z= 4050+ }x—- 3s (10.4e) 


The problem is now reduced to exactly the same form as (10.3), and the same pro- 
cedure can be applied. The non-basic variables are x = s = 0, and the basic variables are 
r = 600, y = 450, t = 250 and u = 1200, and z has increased its value from 0 to 4050. 

Now only x can be increased, since increasing the other non-basic variable, s, would 
decrease z. Increasing x to 500 in (10.4c) and reducing ¢ to 0 is the best that can be done. 
Using (10.4c) to write x = 3s — 2t+ 500, we now eliminate x from the other equations 
to give 


s—2t+r = 100 (10.5a) 
2s-— t +y = 200 (10.5b) 
—35 + 2t +x = 500 (10.5c) 
6s — 4t +u= 200 (10.5d) 
and 
z= 4300 -3s-t (10.5e) 


We now have the final solution, since increasing s or ¢ can only decrease z. Thus we 
have x = 500, y = 200, which is in agreement with the previous graphical solution, the 
maximum profit is z = 4300 as before, and the amounts left over in Figure 10.4 are just 
the 100 and 200 appearing on the right-hand sides of (10.5a, d). 

We have just described the essentials of the simplex algorithm, although the method 
of working may have appeared a little haphazard. It can be tidied up and formalized 
by writing the whole system in tableau form. Equations (10.3) are written with the 
basic variables in the left-hand column, the coefficients in the equations placed in the 
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appropriate array element and the objective function z placed in the first row with minus 
signs inserted. 


Non-basic 
variables Basic variables 
x y r S t u Solution 
Objective function z —5 -9 0 0 0 0 0 
1 3 4 1 0 0 0 2400 
Basic variables J * J . 0 : 0 9 900 
t 2 3 0 0 1 0 1600 
u 2 0 0 0 0 1 1200 


The current solution can easily be read from the tableau. The basic variables in the 
left-hand column are equal to the values in the solution column, so r = 2400, s = 900, 
t= 1600 and w= 1200. The remaining non-basic variables are zero, namely x = y= 0. 
The profit z is read similarly as the entry in the solution column, namely z = 0. The 
negative signs in the z row ensure that z remains positive in the subsequent manipula- 
tion. It should be noted that a 4 x 4 unit matrix (shown shaded) occurs in the tableau 
in the basic variable columns, with zeros occurring above in the z row. This standard 
display is always the starting place for the simplex method, with the only possible 
complication being that the columns of the unit matrix might be shuffled around. The 
algorithm can now be performed in a series of steps: 


Step 1 


Choose the most negative entry in the z row and mark that column (the y column in 
this case). 


Step 2 


Evaluate the ratios of the solution column and the positive entries in the y column, 
choose the smallest of these and mark that row (the s row in this case). 


x y r S t u Solution 
Zz -5 =o 0 0 0 0 0 Ratios 
3 4 1 0 0 0 2400 2400/4 = 600 
1 Q) 0 1 0 0 900 900/2 = 450 
t 2 3 0 0 1 0 1600 1600/3 = 5333 
u 2 0 0 0 0 1 1200 - 


Step 3 


Change the marked basic variable in the left-hand column to the marked non-basic variable 
in the top row (in this case s changes to y in the left-hand column). 
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Step 4 


Make the pivot (the element in the position where the marked row and column cross) 
1 by dividing through. In this case we divide the row elements by 2. These series of 
steps lead to the tableau 


x y r S t u Solution 
Zz —5 -9 0 0 0 0 0 
} 3 + 1 0 0 0 2400 
y 5 1 0 5 0 0 450 
t 2 2 0 0 1 0 1600 
u 2 0 0 0 0 1 1200 


Step 5 


Clear the y column by subtracting an appropriate multiple of the y row (this is just 
Gaussian elimination); for example, (z row) + 9 x (y row), (7 row) — 4 X (y row) and 
so on. This leads to the tableau 


x y r S t u Solution 
Zz -} 0 0 ; 0 0 4050 
r 1 0 1 —2 0 0 600 
y 5 1 0 ; 0 0 450 
t ; 0 0 3 1 0 250 
u 2 0 0 0 0 1 1200 


This tableau can now easily be recognized as equations (10.4). It may be noted that the 
unit matrix appears in the tableau again, with the columns permuted, and the z row has 
zero entries in the basic variable columns. 

The tableau is in exactly the standard form, and is ready for reapplication of the five 
given steps. Steps 1 and 2 give the tableau 


Be y r Ss t u Solution 
z - 0 0 2 0 0 4050 Ratios 
r 1 0 1 2 0 0 600 600 
y i 1 0 4 0 0 450 900 
Ge 0 0 == 1 0 250 500 
u 2 0 0 0 0 1 1200 600 
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Steps 3, 4 and 5 then produce a final tableau (compare with equations (10.5)) 


x y r S t u Solution 
Z 0 0 0 3 1 0 4300 
r 0 0 1 1 —2 0 100 
y 0 1 0 2 -l 0 200 
x 1 0 0 —3 2 0 500 
u 0 0 0 6 —4 1 200 


All the entries in the z row are now positive, so the optimum is achieved. The solution 
is read from the tableau directly; the left-hand column equals the right-hand column, 
giving z = 4300, r = 100, y = 200, x = 500 and wu = 200, which is in agreement with the 
solution obtained in Example 10.3. 


Simplex algorithm: general theory 


We can now generalize the problem to the standard form of finding the maximum of 
the objective function 


Z=CyX, + CyXy t+... 4+C,X, 
subject to the constraints 


QAyX, + AyyxX.+ ... +A, xX, S 


b 
AX, + AyX. +... +a), X, Sb, 


(10.6) 
Any X + Am 2Xo +... + AnyXy S by, 
by the simplex algorithm, where the b,, b,,..., b,, are all positive. By introducing the 
slack variables x4), .- » 5 Xnim = 0 we convert (10.6) into the standard tableau 
Xj xX, se ae Kaa Eres 5 au Mio Solution 
4 -C; =, sete Cc; 0 0 and 0 0 
Keay ay ay) Ai, 1 0 0 b, 
Xnga ay, Ay Ary 0 1 0 b; 
Kix a ha 2 0 ; 1 b,, 


Any subsequent tableau takes this general form, with an m Xx m unit matrix in the basic 
variables columns. As noted in the previous example, the basic variables change, so the 
left-hand column will have m entries, which can be any of the variables, x), .. ©. Xin 
The unit-matrix columns are usually not in the above neat form but are permuted and 
hence the zeros of the ‘z’ row can be any of the | to n+m entries corresponding to the 
basic variables. 
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The five basic steps in the algorithm follow quite generally: 


Step 1 


Choose the most negative value in the z row, say—c;.___ (Identify column 7) 
If all the entries are positive then the maximum has 
been achieved. 


Step 2 


Evaluate b,/a,;, b,/a);,..., b,,/a,,; for all positive ay. (Identify row /) 
Select the minimum of these numbers, say 5;/a;. 


Step 3 


Replace x,,,; by x; in the basic variables in the left- (Change the basis) 
hand column. 


Step 4 


In row j replace a, by a,/a; fork=1,...,n+m+1. (Make pivot = 1) 
(Note that the first row and the final column are treated 

as part of the tableau for computation purposes, —c, = do,, 

b, a Qaintmst):) 


Step 5 


In all other rows, /#/, replace a, by a, —a,a, for allk=1, (Gaussian elimination ) 
eit wluancdstormeac hiro wali —s0sersenenrTen (=97))s 


The algorithm is then repeated until at Step 1 the maximum is achieved. The method 
provides an extremely efficient way of searching through the corners of the feasible region. 
To inspect all corners would require the computation of ("7") points, while the simplex 
algorithm reduces this very considerably, often down to something of the order of m+n. 

Several checks should be made at the completion of each cycle, since it may be 
possible to identify an exceptional case. Perhaps the most complicated of the exceptions 
is when one of the b, = 0 during the calculation, implying that one of the basic variables 
is zero. This can be a temporary effect, in which case the problem goes away at the next 
iteration, or it may be permanent, and that basic variable is indeed zero in the optimal 
solution. The best that may be said, other than going into sophisticated techniques 
found in specialist books on LP, is that problems are possible and the computation 
should be watched carefully. The solution can get into a cycle that cannot be broken. 

A second exception, that should be noted carefully, occurs when one of the c;= 0 for 
a non-basic variable in the optimal tableau. The normal simplex algorithm can then change 
the solution without changing the z row by selecting this 7 column at Step 1. Because 
c; = 0, Step 5 is never used on the z row at all. This case corresponds to a degenerate 
solution with many alternative solutions to the problem, and geometrically the profit 
function is parallel to one of the constraints. 
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The third exception occurs at Step 2 when all the a,;, a), ..., a,,; in the optimal 
column are zero or negative and it becomes impossible to identify a row to continue the 
method. The region in this case is unbounded, and a careful look at the original problem 
is required to decide whether this is reasonable, since it may still be possible to get a 
solution to such a problem. 


Find the maximum of 

z= 5x, + 4x, + 6x, 
subject to 

4x, + x,+ x,< 19 


30 


IN 


3x, + 4x, + 6x; 
2x, + 4x,+ x; S25 
X,+ x»+2x, <= 15 


X1,X), x; 20 


The example cannot be solved graphically, since it has three variables, but is in a correct 
form for the simplex algorithm. The initial tableau gives the solution x, = 19, x; = 30, 
X¢ = 25, x, = 15 and non-basic variables x, = x, =x, =0. 


xy X> x3 X4 Xs X¢6 X4 Solution 
Zz -5 -4 ~6 0 0 0 0 0 Ratios 
X4 4 1 1 1 0 0 0 19 19/1 = 19 
Xs 3 4 ©) 0 1 0 0 30 30/6= 5 
X¢ 2 4 1 0 0 1 0 25 25/1 = 25 
x; i 1 2 0 0 0 1 15 15/2= 7.5 


In the initial tableau the pivot is identified, and x; is removed from the basic variable 
column and replaced by x3. The pivot is made equal to unity by dividing the x, row by 
6. The other entries in the x; column are then made zero by the Gaussian elimination in 
Step 5. This gives the tableau 


X, Xx; X4 Xe X~ xy Solution 
Z 0 0 0 1 0 0 30 Ratios 
OA 2 0 1 a 0 0 14 4 
X3 2 1 0 L 0 0 5 10 
X¢ a 0 0 -1 1 0 20 13.3 
X, -! 0 -! 0 1 5 = 
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The process is then repeated and the pivot is again found, x, is replaced by x, in the 
first column, and the next tableau is constructed by following the remaining steps of the 
simplex algorithm, giving the tableau 


x xX, x3 X4 Xs X¢6 Xy Solution 

z 0 + 0 5 e 0 0 38 

# 1 = 0 2 -* 0 0 4 
3 1 

X3 0 7 1 =5 = 0 0 3 
3 

Xe 0 g 0 5 -2 1 0 14 

x, 0 -} 0 0 -! 0 1 5 


Thus the solution is now optimal, and gives x, = 4, x, = 0, x; = 3, and z = 38 as the 
maximum value. Note that the first two constraints are binding, that is satisfied exactly, 
while the other two are not. This can easily be deduced by looking at the slack variables 
in the initial tableau. We have x, =x; = 0, corresponding to the first two constraints, and 
X56 #0, x, # 0 for the last two constraints. 


Computers are particularly helpful when there is an efficient algorithm, such as the 
simplex algorithm for solving LP problems, since they can perform the arithmetic 
with speed and accuracy. A typical implementation of the algorithm in MAPLE for 
Example 10.4 is now given: 


with(simplex) : 

constri=(4*x14%24x3<=19, 3*x144*x24+6*x3<=30, 
Dy OA SAO 3) = )'5) 7, MEI AEE aE eS) = I'S) fF 

Ole) 8 By ocll seth ee MG oo c8) 0 

maximize (obj, constr,NONNEGATIVE) ; 


These few lines of code give x, =4, x, = 0 and x, = 3 instantly. Similarly in MATLAB, 
LP problems can be solved but are set up in a slightly different way. It always solves 
the minimum problem 
Ax =b 
min,f‘x such that + Aeg = beg 


lb =x =<ub 


and the way that the problem is tackled can be controlled in opt imset. The follow- 
ing lines of code give the solution to Example 10.4: 


t=|—5p=-de—6)¢ A=(4 i ies 4 Ge2 4A ipl 1 Bie lsSllSesOe25e15]|/ 
Aeq=[ ]; beq=[ ]; lb=zeros(3,1); ub=[ ]; x0=[ ]; 
$[ ] indicates not used but the lower bound, 1b, must be 
set to zero 
CMe LOIS Clie MSS (iene Ssoella” , “Orci”, “Simoes” , “ein” )) 2 
[x,fval,exitflag, output, lamda]=linprog(f,A,b,Aeq,beq,1b, 
ub,x0,options) 
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Example 10.5 


Solution 


Typing lamda.ineqglin gives the values 0.5714, 0.9048, 0, 0 which are the 
values in the z-row of the final tableau and determines whether or not the inequalities 
are binding. 

Clearly this is the quick way to get the ‘answer’, but it does not give any under- 
standing of the method. The package has the facilities to go through the steps of the 
algorithm one at a time so it can be used to help with the arithmetic while leaving 
the user to determine the steps of the method. 


A firm has two plants, P1 and P2, that can produce a particular chemical. The product 
is made from three constituents, A, B and C. In a given period there are 36 000 litres of 
A, 30000 litres of B and 12 000 litres of C available. Plant P1 requires the constituents 
A, B, C to be mixed in the ratio 4:2:1 respectively, and the manufacturer makes a profit 
of £1.50 per litre of product; plant P2 requires the ratio 3:3:1, and gives a profit of 
£1 per litre of product. 

Determine how production should be allocated to each plant to maximize the profits, 
and how much of A, B and C remain. 

There is a major breakdown in the supply of chemical C, so that only 8000 litres are 
available in the given period. How should production be changed to maximize the 
profits, how much has profit been reduced, and how much of A, B and C remain? 


For each 1000 litres produced in plant P1, + x 1000 will be constituent A, ? x 1000 


7 


will be B and } x 1000 will be C. For each 1000 litres produced in plant P2, ? x 1000 


25 
will be constituent A, > x 1000 will be B and + x 1000 will be C. Thus, taking the three 
constituents in turn and letting x, and x, represent respectively the amount (in 1000 litre 
units) produced in plants P1 and P2, we obtain 


$x, + 2x, < 36 4x, + 3x, < 252 
$x, + 3x,<30 or 2x,+3x, < 210 
x, = 12 X,+ XS 84 
X1,xX, = 0 
and the profit 
Z= 1.5%, +25 


We can immediately construct the initial tableau 


X> Xs X4 Xs Solution 
Zz =1 0 0 0 0 Ratios 
be 3 1 0 0 250) 63 
X4 3 0 1 0 210 105 
Xs 1 0 0 1 84 84 
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The pivot has been found, and hence we introduce x, into the basis and construct the 
next tableau following the steps of the simplex algorithm: 


x X> x3 X4 Xs Solution 
Zz 0 : 7 0 0 94.5 
x 1 3 i 0 0 63 
X4 0 3 - ; 1 0 84 
Xs 0 : _ 0 1 21 


The z row is all positive, and hence we can immediately read off the solution (multiply 
by 1000 to re-establish proper costs) 


x, = 63 000, x, =0, z= £94500 
and only plant P1 is utilized. From the initial tableau we see that since x, = 0, there are 
zero litres of A remaining; x, = 84, so that we have (84/7) x 1000 = 12 000 litres of B 
remaining; and x; = 21, so that (21/7) x 1000 = 3000 litres of C remain. 
After the breakdown, the 12 000 litres of C are reduced to 8000 litres, so that the first 
tableau becomes 


ier X> % X4 Xe Solution 
Zz =I -1 0 0 0 0 Ratios 
x; 4 3 1 0 0 252 63 
X4 2 3 0 1 0 210 105 
b dd) 1 0 0 1 56 56 


We note that we have a different pivot, and hence we expect a different solution. The next 
tableau is derived in the usual way, giving 


x; X> X, X4 Xs Solution 
Zz 0 0.5 0 0 1.5 84 
X3 0 -1 1 0 —4 28 
X4 0 1 0 1 —2 98 
x, 1 1 0 0 1 56 


The tableau is again optimal, so 
x, = 56000, xX, =0, z= £84000 


The profit is thus reduced by £10 500 by the breakdown, but still only plant P1 is used. 
The remaining amounts of A, B and C can be checked to be 4000, 14000 and zero litres 
respectively. 

Since this problem has only two variables, it would be instructive to check these 
results using the graphical method. 


www.2Ofile.org 


Example 10.6 


Solution 


10.2 LINEAR PROGRAMMING 859 


Find the maximum of 
z=4x, + 2x, + 4x, 
subject to 
3x, +X) + 2x, < 320 
xX, +%+ x, < 100 


2x, +X) + 2x, < 200 


x x, Xe X4 X5 X¢ Solution 
Zz 4 2 4 0 0 0 0 Ratios 
Xq 3 1 2 1 0 0 320 160 
Z l 1 dd) 1 0 100 100 
X~6 2 1 7 0 0 1 200 100 


Note that in the above tableau there is some arbitrariness in the choices in both Steps 1 
and 2. In Step 1 the column is chosen arbitrarily between the x, and x; columns. From 
the ratios, the x; row is selected from the x; and x, rows at Step 2, which both have equal 
ratios. Steps 3—5 are then followed to give the tableau 


58 xX, x3 X4 Xs X6 Solution 
Zz 0 2 0 0 4 0 400 
X4 i -1 1 —2 0 120 
bi 1 1 0 1 0 100 
Xe 0 —1 0 0 —2 1 0 


Although this is the optimal solution with x, = x, = 0, x; = 100 and z = 400, we have 
c, = 0 in the z row. Since x, is a non-basic variable, there is degeneracy. If we follow 
through the algorithm, choosing the first column at Step 1, we obtain an equally optimal 
solution in the following tableau. Replace x; by x, in the basic variables, and subtract 
the x, row from the x, row: 


xy X> X3 X4 Xs X6 Solution 
Zz 0 2 0 0 4 0 400 
x4 0 —2 -1 1 3 0 20 
x, 1 1 1 0 1 0 100 
XG 0 -1 0 0 —2 1 0 


This solution gives x, = 100, x, =x; =0 and z= 400 once more. It can easily be deduced 
that x, = 100(1 — @), x, = 0, x; = 100q@ is an optimal solution for any 0 <a 1 
with z = 400. We could have observed this fact geometrically, since z is just a multiple 
of the left-hand side of the last constraint. 
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10.2.4 Exercises 


Use the graphical method to find the maximum 
value of 


f=4x + 5y 
subject to 


3x+ 7y S 10 
2x+ ys 3 
x,y 20 
Sketch the constraints 
2x- yS6 
x+2yS8 
3x+2y S 18 
ys3 


and verify that the maximum of the function 

x + y in the feasible region is at x = 4 and y =2. 
Check the solution with the simplex method. 
Use a package such as MAPLE or MATLAB to 
verify the solution. 


A manufacturer produces two types of cupboard, 
which are constructed from chipboard and oak 
veneer that both come in standard widths. The first 
type requires 4m of chipboard and 5 m of oak 
veneer, takes 5h of labour to produce and gives a 
profit of £24 per unit. The second type requires 5m 
of chipboard and 2 m of oak veneer, takes 3h of 
labour to produce and gives a £12 profit per unit. 

On a weekly basis there are 400 m of chipboard 
available, 200 m of oak veneer and a maximum of 
250h of labour. Write this problem in a linear 
programming form. Use the simplex method to 
determine how many cupboards of each type should 
be made to maximize profits. How much profit is 
made? Which of the scarce resources remain 
unused? Show that the amount of oak veneer 
available can be reduced to 175 m without affecting 
the basis. What is the new solution, and by how 
much is the profit reduced? 


A factory manufactures nails and screws. The profit 
yield is 2p per kg nails and 3p per kg screws. Three 
units of labour are required to manufacture | kg nails 
and 6 units to make 1 kg screws. Twenty-four units 
of labour are available. Two units of raw material 
are needed to make | kg nails and 1 unit for 1 kg 
screws. Determine the manufacturing policy that 


yields maximum profit from 10 units of raw material. 


A manufacturer makes two types of cylinder, CYL1 
and CYL2. Three materials, M1, M2 and M3, are 
required for the manufacture of each cylinder. 

The following information is provided: 


Quantities of materials required 


Ml M2 M3 
CYL1 1 1 
CYL2 5 2 1 


Quantities of materials available 


Ml M2 M3 


45 21 24 


£4 profit is made on one CYL1 and £3 profit on 
one CYL2. How many of each cylinder should 
the manufacturer make in order to maximize 
profit? 


The Yorkshire Clothing Company makes two 
styles of jacket, the ‘York’ and the ‘Wetherby’. 
The York requires 3 m of cloth and 3h of 

labour, and makes a profit of £25. The Wetherby 
needs 4m of cloth and 2h of labour, and makes 

a profit of £30. The Yorkshire has 400 m of 

cloth available and 300 h of labour available 

each week. Advise the company on the number 

of each style it should produce in order to maximize 
profits. 

The company is prepared to buy more cloth to 
increase its profits, but it will not employ any more 
labour. Under this revised policy, is there a strategy 
that will increase its profits? 


Find the optimal solution of the following LP 
problem: maximize 

z=kx, + 20x, 
subject to 


x, + 2x, = 20 


3x, + XxX = 25 


X1,X, 20 
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where k is a positive parameter representing 
variable profitability. Use both the simplex method 
and the graphical method, and interpret the results 
geometrically. 


Use the simplex method to solve the following 
problem: maximize 


2x, +x, + 4x; 4+ Xx, 


subject to 
2X; + Xx <3 
x 3x, +x, 4 
4x, + xX; +X,S3 


X1, Xp, Xz, Xy = O 


A publisher has three books available for printing, 
B1, B2 and B3. The books require varying amounts 
of paper, and the total paper supplies are limited: 


Total units 


Bl B2 B3 available 


Units of paper 3 2 1 60 
required per 
1000 copies 
Profit per 1000 
copies 


£900 £800 £300 


10.2.5 Two-phase method 


10 


ital 
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Books B1 and B2 are similar in content, and the 
total combined market for these two books is 
estimated to be at most 15 000 copies. Determine 
how many copies of each book should be printed 
to maximize the overall profit. 


Euroflight is considering the purchase of new 
aircraft. Long-range aircraft cost £4 million each, 
medium-range £2 million each and short-range 

£1 million each, and Euroflight has £60 million 

to invest. The estimated profit from each type 

of aircraft is £0.4 million, £0.3 million and £0.15 
million respectively. The company has trained pilots 
for at most a total 25 aircraft. Maintenance facilities 
are limited to a maximum of the equivalent of 30 
short-range aircraft. Long-range aircraft need twice 
as much maintenance as short-range ones, and 
medium-range 1.5 times as much. Set this up as a 
linear programming problem, and solve it. Aircraft 
can only be bought in integer numbers, so estimate 
how many of each type should be bought. 


Find x,, Xx», 3, Xx, = 0 that maximize 


f= 6x, +x, + 2x; + 4x, 


subject to 
2x, + X2 + x, 3 
xy + 434+ mS 4 


xX, +x, + 3x; + 2x, <= 10 


The previous section only dealt with ‘<’ constraints and did not consider ‘=’ constraints. 
These prove to be much more troublesome, since there is no obvious initial feasible 
solution, and Phase | of the two-phase method is solely concerned with getting such 
a solution. Once this has been obtained, we then move to Phase 2. This is the standard 
simplex method, starting from the solution just obtained from Phase 1. A simple example 
will illustrate the problems involved and the basic ideas of the two-phase method. 


Find the maximum of 
z=xt+y 


subject to 
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Figure 10.5 
The feasible region. 


Solution 


The region defined by the constraints is shown in Figure 10.5. It is clear from the figure 
that the origin is not in the feasible region and that x = 4, y = 5 gives the optimal solution. 
We have already appreciated that the graphical method is only useful for two-dimensional 
problems, so we must explore how the simplex method copes with this problem. 

Add in the positive variables r, s and f to give 


—x+2y+r =6 
x +5 =4 
2x+ y —t=4 


The r and s are the usual slack variables. Because we must subtract ¢ to take away the 
surplus, ft is called a surplus variable. The obvious solution x = y = 0, r= 6, s=4, t=—4 
does not satisfy the condition that all variables be positive. The algebra is saying that 
the origin is not in the feasible region. Because the simplex method works so well, the 
last equation is forced into standard form by adding in yet another variable, u, called an 
artificial variable, to give 


2x+y—-tt+u=4 


Now we have a feasible solution x = y= t=0, r= 6, s=4, u=4, but not to the problem 
we originally stated. As the term ‘artificial variable’ implies, we wish to get rid of wu and 
then reduce the problem back to our original one at a feasible corner. The variable u 
can be eliminated by forcing it to zero and this can be done by entering Phase 1 with a 
new cost function 


Z=-u 


We see that if we can maximize z’ then this is at v= 0, and our Phase 1 will be complete. 
The simplex tableau for Phase | then takes the form 


x y r KY t u Solution 
-1 -1 0 0 0 0 0 
: 0 0 0 il 0 
7 -1 2 1 0 0 0 6 
Ss 0 0 1 0 4 
u 2 1 0 0 -1 1 4 
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where z has been included for the elimination but does not enter the optimization: only 
the z’ row is considered in Phase 1. It may be observed that the tableau is not of standard 
form, since u is a basic variable and the (z’, w) entry is non-zero. This must be remedied 
by subtracting the u row from the z’ row to give the standard-form tableau 


DG y r S t u Solution 
Zz =1 =1 0 0 0 0 0 
: —2 -1 0 0 1 0 —4 
1 -1 2 1 0 0 6 
1 0 0 1 0 0 4 
@ I 0 0 =1 1 - 


Manipulation using the usual simplex algorithm gives the tableau 


% y r Ss t u Solution 
Z -! + 3 
z’ 0 0 1 
r 0 ; 1 0 -3 5 8 
5) 0 3 1 ob 4 
x 1 ; 0 = : 


At this stage z’ = 0 and u = 0, so that we have driven wu out of the problem, and the z’ 
row and w column can now be deleted. The Phase | solution gives x = 2, y= 0, which can 
be observed to be a corner of the feasible region in Figure 10.5. 

We now enter Phase 2, with the z’ row and wu column deleted, and perform the usual 
sequence of steps. The initial tableau is 


x y r Ss t Solution 


So 
= 
= 
| 
NIE NI Ne 


i=) 
| 


x y r Ss t Solution 
Zz 0 0 7 0 - 8 
2 1 16 
y l 3 0 Ds 5 
1 18 
Ss z 1 @) esi 
1 2 2 
x “5 0 ns 5 
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Example 10.8 


x y r S t Solution 
Zz 0 0 5 3 0 9 
y ° } 0 
t 5 7 1 
x 0 1 0 


We now have an optimum solution, since all the z row entries are non-negative with x = 4, 
y= 5 and objective function z = 9 in agreement with the graphical solution. 


The general two-phase strategy is then as follows: 


Phase 1 


(a) Introduce slack and surplus variables. 
(b) Introduce artificial variables alongside the surplus variables, say x,, ... , x,. 
(c) Write the artificial cost function 


Bs — — 
Zi IN al Nate ents eX 


(d) Subtract rows x,, x,41,..-,X, from the cost function z’ to ensure there are zeros 
in the entries in the z’ row corresponding to the basic variables. 

(ce) Use the standard simplex method to maximize z’ (keeping the z row as an extra 
row) until z’ = 0 and 


Do ee x, =0 
Phase 2 
(a) Eliminate the z’ row and artificial columns x,, . . . , x). 


(b) Use the standard simplex method to maximize the objective function z. 


There are other approaches to obtaining an initial feasible basic solution, but Phase | of the 
two-phase method gives an efficient way of obtaining a starting point. Geometrically, 
it uses the simplex method to search the non-feasible vertices until it is driven to a 
vertex in the feasible region. 


Use the two-phase method to solve the following LP problem: maximize 
z=4x,+ bX» +X 
subject to 
x, + 2x, + 3x, 22 
2x, + X%+ x,<5 


X15 X2, X3 20 
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Solution 
Phase I Introduce a surplus variable x, and a corresponding artificial variable x; into the first 
inequality. A slack variable x, is required for the second inequality. The artificial cost is just 


xy XA x3 X4 Xs X6 Solution 
Zz t 0 
z’ 0 0 1 
Xe iE 2 3 -1 1 0 2 
X6 2 1 1 0 0 1 5 


We subtract the x; row from the z’ row to eliminate the 1 from the (z’, x;) element, 
5 5 


giving the tableau 
xy X> Be X4 Xs X6 Solution 
Zz 4 t 1 0 0 
z 2 3 1 —2 
be 1 2 @) =1 I 0 2 
X¢ 2 1 1 0 0 1 5 


x XS X3 X4 X5 Xe Solution 
ait L al l 2 
2 3 6 3 3 3 
zg 0 0 0 1 0 
1 2 1 l 2 
x3 3 3 1 “3 3 0 3 
5 1 l i 13 
X6 3 3 0 3 3 l 3 


Since z’ = 0 and the artificial variable x; has been driven into the non-basic variables, 
phase | ends. 


Phase 2 Thez’ row and the x; column are now deleted, and the following sequence of tableaux 
constructed following the rules of the simplex algorithm: 


ber x x3 X4 X6 Solution 
al l cork 2 
z 3) 6 0 3 0 3 
2 at 2 
pes @ 3 l 3 0 3 
a 1 1 13 
Xo 3 3 0 3 : 3 
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Example 10.9 


Xi XS X3 Se X~6 Solution 
Zz 0 7 11 —4 0 8 
Xi 1 2 3 —1 0 2 
Xp 0 =3 =§ Q) 1 1 

Xi X> X3 X4 X~6 Solution 
Zz 0 3 1 0 2 10 
x 1 ; 5 0 ; 3 
mn | 0 2 -$ 1 3 | 


The solution is now optimal, with x, = 2. xX, = x; = 0 and z = 10. Note that the first 
inequality is not binding, since x, # 0, while the second inequality is binding. 


As indicated in the previous section a computer package such as MAPLE or MATLAB 
can deal easily with LP problems. For ‘>=’ inequalities the packages are equally efficient 
and will produce the answer, but there is no indication that the two-phase method has 
been used. For Example 10.8 the MAPLE code 


with(simplex): 
Constr s={x142* x43 *x3Se2 ,2* xlsx 5 ea5): 
Obgi = Ar xd 5402 xo 

maximize (obj,constr,NONNEGATIVI 


i 
produces the result x, = 5/2, x, = 0 and x, = 0 instantly. 
The corresponding MATLAB code is 


E 


t=(=4 9-0, S71] 9 ASl=1 =2 =392 1 Iie b=[=2,5]% 

MeG= |e loeG=|l lp Mosveros (4,1) 7 Weel |pev= IF 

options=optimset (‘LargeScale’,‘off’,‘’Simplex’,’on’); 

[x,fval,exitflag, output, lamda]=linprog(f,A,b,Aeq,beq,1b, 
ub, x0,options) 


Three ores, A, B and C, are blended to form 100 kg of alloy; the percentage contents and 
the costs are as follows: 


Ore A B Cc 

Iron 70 60 0 
Lead 20 10 40 
Copper 10 30 60 
Cost (£kg™') 3000 2000 1000 


The alloy must contain at least 20% iron, at least 25% lead but less than 48% copper. 
Find the blend of ores that minimizes the cost of the alloy. 
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Let x,, x, and x, be the weights (kg) of ores A, B and C respectively in the 100 kg of alloy. 
The constraints give 


iron 0.7x, + 0.6x, = 20 
lead 0.2x, + 0.1x, + 0.4x; = 25 
copper 0.1x, + 0.3x, + 0.6x; < 48 
and to make the 100 kg of alloy, 
X, +X, +x; = 100 
The cost is 
3000x, + 2000x, + 1000x, 


which is to be minimized. 
To reduce the problem to standard form, we change the problem to a maximization of 


z=—3000x, — 2000x, — 1000x; 


For the inequality constraints we use surplus variables x, and x; and a slack variable x,. 
We require two artificial variables x, and x, alongside the surplus variables. Thus the 
inequalities become 


0.7x, + 0.6x, — X4 +X; = 20 
0.2x, + 0. Lx, + 0.4x; — Xz +X, = 25 
0.1x, + 0.3x, + 0.6x; +X = 48 


To deal with the equality constraint, we introduce a further artificial variable x,: 
X, +X) +X; +X = 100 


We must drive x, to zero, to ensure that the equality holds, so it is essential to put x, 
into the artificial cost function. (Note that this is the standard way of dealing with an 
equality constraint.) We first enter Phase 1. Steps (a)—(c) of Phase | of the two-phase 
method give the initial tableau 


xy X> X3 X4 Xs X6 x4 Xe Xo Solution 
Zz 3000 2000 1000 0 0 0 0 0 0 0 
z’ 0 0 0 0 0 0 1 1 1 0 
x, 0.7 0.6 0 -1 0 1 0 0 20 
Xe 0.2 0.1 0.4 0 -1 0 0 1 0 25 
Xs 0.1 0.3 0.6 0 0 1 0 0 0 48 
Ke 1 1 1 0 0 0 0 0 1 100 


It is necessary to remove the Is from the z’ row in the basic variable columns x,, x, and 
X . Following (d) of the general strategy, we replace the z’ row by (z’ row) — (x, row) — 
(xg row) — (Xx) row) to give the tableau 
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Xi X> Xe x Ke Xe xX; xy XG Solution 
Zz 3000 2000 1000 0 0 0 0 0 0 0 
z -1.9 -1.7 -14 1 1 0 0 0 0 —145 
xX, 0.7 0.6 0 -1 0 1 0 0 20 
Xg 0.2 0.1 0.4 0 -1 0 0 1 0 25 
Xe 0.1 0.3 0.6 0 0 1 0 0 0 48 
Xo 1 1 1 0 0 0 0 0 1 100 


Several tableaux need to be completed to drive z’ to zero and complete Phase 1, with 


the final tableau being 

xX; XX Xz; X, Xs Xs XX, Xe Xo Solution 
Zz 0  -2000 0 0 -10000 0 0 10000 —1000 —250 000 
z | 0 0 0 0 0 0 1 1 1 0 
x, | 1 15 0 0 5 0 0 —5 2 75 
x; | 0 -0.5 1 0 —5 0 0 Bi -1 25 
xX, | 0 045 0 O 25° -1 0 2.5 0.4 25.5 
x, | 0 045 0 1 35 0 -l 3.5 1.4 32.5 


Removing the artificial variables and the z’ row gives the tableau 


Xx) X> X3 X4 Xs X6 Solution 
Zz 0 —2000 0 0 —10000 0 —250.000 
x 1 1.5 0 0 5 0 75 
x3 0 —0.5 1 0 —5 0 25 
X6 0 0.45 0 0 25 1 25,5 
x4 0 0.45 0 1 BS 0 32.5 


The algorithm is now ready for Phase 2, since a sensible feasible basic solution is 
available. The standard procedure leads, after many cycles, to the final tableau 


xX, X> X3 X4 Xs X6 Solution 
Zz 333.3 0 0 0 0 3333.3 —140000 
X4 0.3 0 0 1 0 —2 4 
X3 —0.67 0 1 0 0 3.33 60 
Xy 1.67 1 0 0 0 3.33 40 
Xs -0.3 0 0 0 1 1 3 


and the solution can be read off as 


x, = 0, x, = 40, x; = 60 
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with the cost minimized at £140 000. (Note that the cost in the tableau is negative, since 
the original problem is a minimization problem.) It may be noted that x, = 0, so the 
copper constraint is binding while the iron constraint gives 4% more than the required 
minimum and the lead constraint gives 3% more than the required minimum. 


Except for simple, illustrative examples, the amount of computational work in the two- 
phase strategy is heavy and requires the use of a computer package. Even for the com- 
paratively simple Example 10.9, many tableaux were required in the solution. The computer 

packages MAPLE and MATLAB have no difficulty in dealing with the equality constraint 
that appears in the problem. 


10.2.6 Exercises 


Use the graphical approach to solve the LP problem Determine the daily production plan to maximize 
profits. Use the two-phase method to obtain the 
max(x + 2y) solution, and verify your result with a graphical 
solution. 


subject to the constraints 


15 ~—_‘InExercise 9 there is an additional union agreement 

that at least 50 000 books must be printed. Does the 
x+y 55 solution change? If so, calculate the new optimum 
strategy. 


lsys4 


Check your solution using the two-phase method 
and by using a MAPLE or MATLAB implementation. 16 Solve the LP problem 


Use the simplex method to find positive values of x, ie max(x + y + 2) 


and x, that minimize subject to the constraints 


f= 10x, +x, ge 
subject to 
x+2y 53 
4x, +x, 32 
y+3zs4 


2x, +x, = 12 
by using the two-phase method. Check your result 
2x,-%,S 4 using MAPLE or MATLAB. 


aby ha 17 Solve the following LP problem: minimize 


Sketch the points obtained by the simplex method 
on a graph, indicating how the points progress 
through Phases 1 and 2 to the solution. 


2x, + 7x, + 4x, + 5x, 


subject to 


The Footsie company produces boots and shoes. X17 %3—X4 = 0 
If no boots are made, the company can produce a 


: : : ‘ +x, 22 
maximum of 250 pairs of shoes in a day. Each pair es 


of boots takes twice as long to make as each pair of X15 XppXq, Xy = 0 

shoes. The maximum daily sales of boots and shoes 

are 200, but 25 pairs of boots must be produced to 18 A trucking company requires antifreeze that 
satisfy an important customer. The profits per pair contains at least 50% of pure glycol and at least 
of boots and shoes are £8 and £5 respectively. 5% of anticorrosive additive. The company can 
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Ig) 


buy three products, A, B and C, whose constituents 
and costs are as follows: 


A B C 
% glycol 65 25 80 
% additive 10 3 0 


Cost (£/litre) 1.8 0.9 1.5 


What blend will provide the required antifreeze 
solution at minimum cost? What is the cost of 
100 litres of solution? 


A builder is constructing three different styles of 
house on an estate, and is deciding which styles 

to erect in the next phase of building. There are 
40 plots of equal size, and the different styles 
require 1, 2 and 2 plots respectively. The builder 
anticipates shortages of two materials, and estimates 
the requirements and supplies (in appropriate 
units) to be as follows: 


Requirements 
Total 
Style 1  Style2 Style3 supply 
Facing 1 2 5 58 
stone 
Weather 3 2 1 72 
boarding 


The local authority insists that there be at least 
5 more houses of style 2 than style 1. If the profits 
on the houses are £1000, £1500 and £2500 for 


Lagrange multipliers 


10.3.1 Equality constraints 


20 


styles 1, 2 and 3 respectively, find how many of 
each style the builder should construct to maximize 
the total profit. 


A manufacturer produces three types of carpeting, 
Cl, C2 and C3. Two of the raw materials, M1 

and M2, are in short supply. The following table 
gives the supplies of M1 and M2 available (in 
1000s of kg), the quantities of M1 and M2 required 
for each 1000 m? of carpet, and the profits made 
from each type of carpet (in £1000s per 1000 m’): 


Quantities 
required 
M1 M2 Profit 
Cl 1 1 2 
C2 1 1 3 
C3 1 0 0 
Quantities available 5 4 


Carpet of type C3 is non-profit-making, but is 
included in the range in order to enable the 
company to satisfy its customers. The company 
has policies that require that, if x,, x, and x, 1000s 
of m? of Cl, C2 and C3 respectively are made then 


x, 21 
and 
X,—X, +x, #2 


How much carpet of each type should the company 
manufacture in order to satisfy the constraints and 
maximize profits? 


In Section 10.2 we looked at the situation where all the functions were linear. As soon 
as functions become nonlinear, the problems become very much more difficult. This 
is generally the case in most of mathematics, and is certainly true in optimization 


problems. 
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multiplier problem. 


Example 10.10 
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Contours of 
f(x, y) = constant 


In Section 9.7.4 of Modern Engineering Mathematics it was shown how to use Lagrange 
multipliers to solve the problem of the optimization of a nonlinear function of many 
variables subject to equality constraints. For the general problem it was shown that the 
necessary conditions for the extremum of 

S(%1, Xn, +665 Xn 
subject to 


ZAX),xX,..-.,xX,)=0 G=1,2,...,m(m<n)) 


are 
OF 4 281 4 9,882 4 4 4 C82 = 0 (k=1,2,...,7) (10.7) 
OX, Ox, Ox, Ox, 

where A,,A5,..., A,, are the Lagrange multipliers. These 1 equations must be solved 


together with the m constraints g; = 0. Thus there are n + m equations in the n + m 
unknowns x,,%),...,X, andA,,A,,..., Am 
For a two-variable problem of finding the extremum of 


f(x, y) subject to the single constraint g(x, y)=0 


the problem is illustrated geometrically in Figure 10.6. We are looking for the 
maximum, say, of the function f(x, y), but only considering those points in the plane 
that lie on the curve g(x, y) = 0. The mathematical conditions look much simpler of 
course, as 


f,+Ag,=0 
f+ Ag, =0 (10.8) 
g=0 
There are two comments that should be noted. First, the method fails if g, = g, = 0 
at the solution point. Such points are called singular points: fortunately they are rare, 
but their existence should be noted. Secondly, sufficient conditions for a maximum, a 
minimum or a saddle point can be derived, but they are difficult to apply and not very 
useful. Example 10.12 will illustrate an intuitive approach to sufficiency. 


A few examples should be enough to remind the reader of the problems involved and 
to show the techniques required to solve the equations. 


Fermat stated in 1661 that ‘Light travels along the shortest path’. Find the path that 
joins the eye to an object when they are in separate media (Figure 10.7). 


www.2Ofile.org 


872 OPTIMIZATION 


Figure 10.7 Fermat’s 
shortest-path problem. 


Solution 


Example 10.11 


Solution 


Y | Medium 1 


' 

at velocity of light =v 
a 
a 


fy 

5 

Medium 2 iB : 
velocity of light = V ’ 
i] 


< L "A 


The velocities of light in the two media are v and V. The time of transit of light is given 
by the geometry as 
ee ce b 
veosa Vcos B 


and is then subject to the geometrical constraint that 
L=atana+ btanp 


Applying (10.8), 


oat 4428 = 2 secortan a+ Aa seca 
0a oa v 


97, 0g _b 2 
a Ta 7 Sec tan B+ Ab sec B 


These give as the only solution 
sin @= —Av, sin B=-AV 


or 


sin@ Vv _ 
sinB V 


which is known as Snell’s law. 


L 


In Example 10.10, to obtain Snell’s law, the solution of the equation was quite straight- 
forward, but it is rarely so easy. Frequently it is technically the most difficult task, and 
it is easy to miss solutions. We return to the ‘milk carton’ problem discussed earlier to 
illustrate the point. 


Find the minimum area of the milk carton problem stated in Example 10.2 and illustrated 
in Figure 10.1. 


Taking measurements in millimetres, the basic mathematical problem is to minimize 
A=(2b+2w+5)(h+b+ 10) 
subject to 


hbw = 1 136000 
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Solution 


Figure 10.8 Hopper. 
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Applying the Lagrange multiplier equations (10.7), we obtain 
0 =(2b+ 2w+5) + Abw 
0= (26+ 2w+5)+2(h+6+ 10)+Ahw 
0= 2(h+b+10)+Ahb 


giving four equations in the four unknowns h, b, w and A. If we eliminate A and w from 
these equations, we are left with the same equations that we derived in Example 10.2, 
which have no simple analytical solutions. 


The only way to proceed further with Example 10.11 is by a numerical solution. Thus, 
even with simple problems such as this one, we need a numerical algorithm, and in most 
realistic problems in science and engineering we encounter similar severe computa- 
tional difficulties. It is often a problem even to write down the function or the constraints 
explicitly. Such functions frequently emerge as numbers from a complicated computer 
program. It is therefore essential to look for efficient numerical algorithms to optimize 
such functions. This will be the substance of Section 10.4. 

A final example shows the situation where there are more than two variables involved. 
An indication will be given of the difficulties of proving that the point obtained is a 
maximum. 


A hopper is to be made from a cylindrical portion connected to a conical portion as 
indicated in Figure 10.8. It is required to find the maximum volume subject to a given 
surface area. 
We can compute the volume of the hopper as 

V=mRL + i nR tana 
and find its maximum subject to the surface area being given as 

A=2nRL + WR’ sec 


Applying (10.7) with the appropriate variables, we obtain 


OV 0g | 2 
0= apt’ OR = 2nRL + WR tana + A(2mL + 20R sec ) 
WV, Og __» 
2mR 
7 tha = mR? + A2n 


0= ov + az = ' mR’ sec’ + AMR’ sec crtan a 
da 0a 


First, 2 can be easily evaluated as A = —}R. The last of the above three equations 
becomes 


TR’ sec’ar(+R + Asin @) = 0 
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and hence sin a@= =. Since 0 < a < }m, we have a= 0.730 rad or 41.8°. A little further 
algebraic manipulation between the constraints and the above equations gives R* = 4/5, 
so that R = 0.3774"? and V = 0.1264°”. 

Normally it would be assumed that this is a maximum for the volume on intuitive 
or geometrical grounds. To prove this rigorously, we take small variations around the 
suspected maximum and show that the volume is larger than all its neighbours. For 
simplicity let R* = (A/m5)(1+ 6), sec a = (3/\/5)(1 + €), evaluate L from the constraint 
g=0 and then calculate V to second order in € and 6 by Taylor’s theorem. Some careful 
algebra gives 

3/2 
y= PETG =i 2g) 
This shows that for any non-zero values of € and 6 we obtain a smaller volume, and 
hence we have proved that we have found a maximum. 


It should be reiterated that the major problem lies in solving the Lagrange multiplier 
equations and not in writing them down. This is typical, and supports the need for good 
numerical algorithms to solve such problems; they only have to be marginally more 
difficult than Example 10.12 to become impossible to manipulate analytically. 


Inequality constraints 


Although we do not intend to consider them in any detail here, for reference we shall 
state the basic extension of (10.7) to the case of inequality constraints. Kuhn and Tucker 
proved the following result in the 1940s. 

To maximize the function 


SQ, oes » Xn) 
subject to 
BK, + 5%) SO G=1,...,m) 
the equivalent conditions to (10.7) are 
Of 4 Or _ ee) Wm _ 6 k=1 
a. te ee te ( peng) 
Aig: = 0 
A,20;7 (G=l1,...,m) 
g <0 


The equation A,g, = 0 gives two alternative conclusions for each constraint, either 


(a) g;=0, in which case the constraint is ‘active’ and the corresponding A; > 0, or 
(b) A; = 0 and g; < 0, so that the optimum is away from this constraint and the 
Lagrange multiplier is not necessary. 


Implementation of the Kuhn—Tucker result is not very easy, even though in prin- 
ciple it looks straightforward. There are so many cases to check that it becomes very 
susceptible to error. 
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10.3.3 Exercises 


Find the optimum of f= x? + xy + y’ subject to the 
constraint x + y = | using Lagrange multpliers. 
Show that the optimum is a minimum. 


Find the shortest distance from the origin to the ellipse 


Determine the lengths of the sides of a rectangle 
with maximum area that can be inscribed within 
the ellipse 


Find the optimum of f= xy’z subject to x + 2y + 3z 
= 6 using a Lagrange multiplier method. 


Show that the stationary points of f= x? + y? +z? 
subject to x + y—z=0 and yz + 2zx — 2xy = | are 
given by the solution of the equations 


0=2x+A+ (22 -2y)u 
0=2y+A+(z—-2x)u 


0=2z7-A+(y + 2x)U 


Add the last of these two equations, and show 
that either u = —2 or y+ z=0. Hence deduce the 
stationary points. 


A rectangular box without a lid is to be made. 

It is required to maximize the volume for a given 
surface area. Find the dimensions of the box when 
the total surface area is A. 


(Harder) The lowest frequency of vibration, 
a, of an elastic plate can be computed by 
minimizing 


Hill climbing 


10.4.1 Single-variable search 


28 
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I[o] = \| (Va)? dx dy 


R 


subject to 


|Jeeo= 


R 


over all functions @(x, y), where R is the region 
of the plate in the (x, y) plane. If R is the square 
region |x| < 1, |y| S 1 and the plate is clamped at 
its edges, use the approximation 


= 21 21 
@= A cos’ 51x cos’ 5 Ty 


to show that [[@pin] = 07 = $ 1°. 


Use the improved approximation 
@ = cos’inx cos’ ny(A + Boos } 1x cos } my) 


to get a better estimate of a. 

(Note: J! cos” } mz dz = (2n)!/(n!)°2”"" for 
non-negative integers n. Preferably use an algebraic 
symbolic manipulator, for example MAPLE, to 
evaluate the differentials and integrals.) 


Use the Kuhn—Tucker criteria to find the minimum of 
2x74 +3 + 2x)xy 


subject to 


where @ is a parameter. Find the critical value 
of a at which the nature of the solution changes. 
Sketch the situation geometrically to illustrate 
the change. 


Most practical problems give calculations that cannot be performed explicitly, and need 
a numerical technique. Typical cases are those in Examples 10.1 and 10.11, where the 
final equations cannot be solved analytically and we need to resort to numerical methods. 
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Figure 10.9 
Bracketing procedure. 


O aa, a as a4 as 


This is not an uncommon situation, and hill climbing methods were devised, mainly in 
the 1960s, to cope with just such problems. 

In many engineering problems the functions that we are trying to optimize cannot be 
written down explicitly. Take for example a vibrational problem where the frequencies 
of vibration are calculated from an eigenvalue problem. These frequencies will depend 
on the parameters of the physical system, and it may be necessary to make the largest 
frequency as low as possible. To illustrate this idea, suppose that the eigenvalues come 
from the equation 


a-2zA -1 0 
1 a 1 |/=0 
OG =) gH? 


where there is just one parameter a. In this case the mathematical problem is to find 
min(A,,ax). We note that there is no explicit formula for /,,,, aS a simple function of a, 


and our function is the result of a solution of the determinantal equation. For some values 
of a the function can be calculated easily, Aimax(—1) = 3, Amax(O) = \2 and Ajna(1) = 2, 
but any other value requires a considerable amount of work. However, a computer 
package such as MATLAB will perform this hard work very quickly with the instruction 
lamda = max(eig([a -1 0; -1 0 -1; 0 -1 a%*2])). Calculating the derivative 
of the function with respect to a is too difficult even to contemplate. 

In the determinant example we have a function of a single variable, but in most problems 
there are many variables. One of the commonest methods of attack is to obtain the 
maximum or minimum as a sequence of single-variable searches. We choose a direction 
and search in this direction until we have found the optimum of the function in the chosen 
direction. We then select a new direction and repeat the process. For this to be a successful 
method, we need to be able to perform single-variable searches very efficiently. This 
section therefore deals with single-variable problems, and only then in Section 10.4.3 
are multivariable techniques discussed. In deciding on a strategy for solution, one crucial 
point is whether derivatives can or cannot be calculated. In the eigenvalue problem, 
calculation of the derivative is difficult, and would probably not be attempted. If the 
derivative can be obtained, however, more information is available, and any numerical 
method can be speeded up considerably. With the increase in sophistication of computers, 
this is becoming a less important consideration, since a good numerical approximation to 
the derivative is usually quite satisfactory. This is certainly the case in the MATLAB 
routine fminunc. 

The basic problem is to determine the maximum of a function y = f(x) that is difficult 
to evaluate and for which the derivative may or may not be available. The task is per- 
formed in two stages: in Stage | we bracket the maximum by obtaining x, and x, such 
that x; S Xpax xX, as described in Figures 10.9 and 10.10, and in Stage 2 we devise 
a method that iterates to the maximum to any desired accuracy, as in Figures 10.11 
and 10.12. 


ks 4h 8h 16h 


«¥ 
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ets ie de for (a) {Derivative not known} 
obtaining a bracket for If a aan is defined by the anonymous function 
: = 09) cco 

the oan ora then the following code 
aold= -1;h=.01;nmax=10; % aold, h, nmax are specified at values appropriate to the problem 
zold=f(aold);a=aold+h;h=2*h;z=f(a);n=0;% step 1 
while (z>zold)&(n<nmax) 

n=n+1;h=2*h;aoldold=aold;aold=a;a=a+h;zoldold=zold;zold=z;z=f(a); % subsequent steps 
end 
provides the bracket [aoldold,aold,a]. 


(b) {Derivative known} 

If the function and its derivative are defined by the anonymous functions 
_OC)in 
fdash=@(x) . ... 

then the following code 
a=.01;h=.01;nmax=10; zold=f(a);zdashold=fdash(a);aold=a;a=a 
+h;z=f(a);zdash=fdash(a);h=2*h;n=0; while (zdash>0) & (n<nmax) 

n=n+1;zold=z;zdashold=zdash;aold=a;z=f(a);zdash=fdash(a);a=at+h;h=2*h; 

end 

provides the bracket [aold,a]. 


A bracket is most quickly achieved by starting at a given point, taking a step in the 
increasing direction and proceeding in this direction, doubling the step length at each step 
until the bracket is obtained (Figure 10.9). The basic idea is summarized in Figure 10.10, 
which gives a MATLAB procedure for this technique. It is written as a ‘stand-alone’ 
segment and can be adapted for other packages, but it would normally be incorporated in 
a more general program. It is assumed that sensible values for a and h have been chosen, 
but in a working program great care has to be taken. A great deal of effort is required to 
cope with inappropriate choices and to prevent the program aborting. 

The algorithm works very efficiently provided that appropriate safeguards are included, 
but it is not foolproof. The maximum number of steps chosen, nmax, is usually 10, and 
the initial value of / is small compared with the overall dimension of the problem under 
consideration. 


Example 10.13 Find a bracket for the first maximum of f(x) =x sin x using the algorithm in Figure 10.10. 


Solution Choose a= 0.01 and f= 0.01; the algorithm then gives 


a 0.01 0.02 0.04 0.08 0.16 0.32 0.64 1.28 2.56 5.12 


f | 0.000 0.000 0.002 0.006 0.025 0.101 0.382 1.226 1.406 —4.701 
f’ | 0.020 0.040 0.080 0.160 0.317 0.618 1.111 1.325 -1.590 — 


If the derivative is not used then the bracket is 1.28 < x = 5.12. 
If the derivative is used then the bracket is 1.28 < x < 2.56. 
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Figure 10.11 
(a) MATLAB =) 


file gapp.m for 

the quadratic 
approximation 
algorithm; the function 
segment for f(x) 

is declared in the 

file fnn.m; (b) diagrams 
corresponding to the 
four cases considered 
in the program. 


Figure 10.12 
(a) MATLAB 2) 


file cufit.m for the 
cubic approximation 
algorithm for the 
maximum of f(x); 

x and the function 
segments f(x) and 
fdash(x) are declared 
in the file cub.m; 

(b) diagrams 
corresponding to the 
two cases considered 
in the program. 


function [x,f]=qapp(a,b) 
% a=[al,a2,a3] is the input vector of three points from bracketing 
% b=[f(a1),f(a2),f(a3)] is the vector of function values 
x=a;f=b;p=polyfit(a,b,2); 
xstar=-0.5*p(2)/p(1);fstar=fnn(xstar); 
if fstar>b(2) 
if xstar<a(2), x(3)=a(2);£(3)=b(2); 
else x(1)=a(2);f(1)=b(2); end 
x(2)=xstar;f(2)=fstar; 
else 
if xstar<a(2), x(1)=xstar;f(1)=fstar; 
else x(3)=xstar;f(3)=fstar; end 


end 
% x contains the three points of the new bracket and f the function values 
(a) 
' ' 
. t 
i] t 
t ty 
iy iy 
‘ iT 
1 ty 
' ‘ 
' 1 
Old labels Xp OXF Xx X3 xy Mg SH NG 

New labels Xp Xy Xs XX, Xs 
i 
a 
t 
i 
‘ 
1 

Old labels xX, xX* x Xs 
New labels X, Xa x3 x X23 


(b) 


function [an,bn]=cufit(a,b) 

% a=[x1 f(x1) fdash(x1)] and b=[x2 f(x2) fdash(x2)] are the input vectors 
v=[a(2);b(2);a(3);b(3)]; 

A=[a(1)*3 a(1)42 a(1) 1;b(1)*3 b(1)*2 b(1) 1;3*a(1)42 2*a(1) 1 0;3*b(1)%2 2*b(1) 1 0); 
p=A\v;xstar=(-p(2)-sqrt(p(2)*2-3*p(1)*p(3)))/(3 *p(1));e=cub(xstar); 

if c(3)>0 an=c;bn=b; else bn=c;an=a;end 

% an and bn contain the new bracket vectors 


(a) 


Old labels x x* Xy 
New labels xy Xy x X> 
(b) 
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Solution 
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Stage 2 of the calculation is to use the bracket just obtained and then iterate to an 
accurate maximum. A simple and efficient approach is to use a polynomial approxima- 
tion to estimate the maximum and then choose the ‘best’ points to repeat the 
calculation. 

If it is assumed that no derivative is available then a bracket is known from the 
algorithm of Figure 10.10, so that x,, x,, x, and the corresponding ff, f, 4, with f, <f, 
and f, < f,, are given. The quadratic polynomial through these points can be written 
down immediately: it is just the Lagrange interpolation formula that was discussed 
in Section 2.3 of Modern Engineering Mathematics. It can easily be checked that the 
quadratic which passes through the required points is given by 


fee ee, Ye, (10.9) 


(x3 — X,)(X3 - x2) ° (x, — X2)(X; - X53) 


+ (x = X3)(% = x1) : 
(x2 - X3)(X2 - x1) 


Then F’ = 0 at the point x*, which is given, after a little algebra, by 


a 05 -D)A+tO3-DA+tQ- DA 


. 2[ (x2 — x3) fi + (%3 — HY) Ao + (%1 - 12) Ai] 


A MATLAB procedure for the algorithm that uses this new x* and /* is given in 
Figure 10.11, where the best three values are chosen for the next iteration. The code can 
be easily adapted for other packages. It can be re-written in terms of anonymous func- 
tions as in Figure 10.10 but there is great merit in breaking the program into small units 
that can be checked independently; these are M-files in MATLAB as in Figure 10.11. 

The method works exceptionally well by repeating the instruction [ a, b] =qapp( a, b) , 
but again it is not totally foolproof, and remedial checks need to be put into a working 
program. The stopping criterion is very problem-dependent, and requires thought and 
numerical experimentation. The MATLAB procedure fminbnd uses the quadratic 
approximation method (for the minimum problem). It only requires a bound for the 
minimum and uses another method, the Golden section (see Exercise 34), to obtain 
three starting values. It then proceeds similarly to the current algorithm. The two lines 
of code below solve the eigenvalue problem posed at the start of this section 


(10.10) 


options=optimset (‘display’,’iter’); 
[x, fval]=fminbnd(‘max(eig([x -1 0;-1 0 -1;0 -1 x%*2]))’, 
-1,1,options) 


It is worth looking at the full code of the MATLAB procedure for fminbnd to appre- 
ciate the enormous effort required to automate the problem fully, to deal with errors and 
failures and to make the progam ‘user friendly’. 


Find the first maximum of f(x) = x sin x given the values from Example 10.13, namely 
x, = 1.28, x, = 2.56, x, = 5.12, f, = 1.226, f, = 1.406 and f, = —4.701. 


From (10.10), x* = 2.03 and f* = 1.820, so for the next iteration choose 
x, = 1.28, X, = 2.03, xX; = 2.56 
f, = 1.226, f= 1.820, f,= 1.406 
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Example 10.15 


Solution 


From (10.10), x* = 1.98 and f* = 1.816, so for the next iteration choose 
x, = 1.98, X, = 2.03, X3 = 2.56 
Ff, = 1.816, = 1.820, f,=1.406 
From (10.10), x* = 2.027 and f* = 1.820, so the method has almost converged. 


When the derivative is available, a better approximating polynomial than (10.9) 
can be used, since x), f,, fj (> 0), x2, f, and 3 (< 0) are known from the bracketing 
algorithm, and this data can be fitted to a cubic polynomial 


f = F=ax'+bx?+cx+d 
F’ = 3ax*+ 2bx +c 


In this case fitting the values just gives the matrix equation 


7 x x x, Illa 
3 2 
So|—| %2 2 % Ii) b (10.11) 
fi 3x, 2X4 1 0 c 
fo|. axe 2 To Ulla 


and the maximum of F is given by F’ = 0, so that 


2 12 
xe = xb + (b' = 3ac) | (10.12) 
3a 


and the negative sign is chosen (the positive sign for a minimum problem) to ensure 
that x, << x* <x). 

A simple algorithm uses these results to choose the appropriate bracket for the 
next iteration. The algorithm is illustrated in Figure 10.12, and is an efficient iterative 
way of evaluating the maximum; just repeat the instruction [a,b]=cufit(a,b). 
Unfortunately it is very easy to make errors in a hand computation, and many people 
prefer the quadratic algorithm for this purpose. On a computer, however, the cubic 
approximation method is almost universally used. 


Find the first maximum of f(x) = x sin x given the bracket values from Example 10.13, 
namely x, = 1.28, f, = 1.226, f{ = 1.325, and x, = 2.56, f5 = 1.406, f5 = -1.590. 


Solving (10.11) gives a=—0.3333, b= 0.7814 and c= 0.9630, and hence, from (10.12), 
x* =2.036, f*=1.820, f*’ =—0.0192 


Thus x5, f; and f5 are replaced by x*, f* and f*’, and x,, f, and fj are retained. 
Equation (10.11) is now recomputed and solved to give 


a= -0.4626, b=1.412, c=-0.0153 
Hence, from (10.12), 
x*=2.029, f*= 1.820, f* =—0.0007 
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Further iterations may be performed to get an even more accurate value. Comparison 
with Example 10.14 shows that both the quadratic and cubic algorithms work well for 


this function. 


&) There are ‘built in’ maximizing routines included in most computer packages 
which can deal with most functions that arise in engineering computations. In MATLAB 
the single instruction fminbnd(‘’-x*sin(x)’,1.28,5.12) produces the value 


x = 2.0288 instantly. 


10.4.2 Exercises 


(a) Find a bracket for the minimum of the function 32 


f(x) =x + 1/x*. Start at x = 0.1 and h = 0.2. @ 


(b) Use two cycles of the quadratic approximation 
to obtain an estimate of the minimum. 

(c) Use one cycle of the cubic approximation to 
obtain an estimate of the minimum. 


The function 


_ sinx 
7 2 
l+x 


SS 


has been computed as follows: 


x —0.5 0 1 3 
ft —0.3825 0 0.4207 0.0141 
f 0.3952 1 —0.1506 —0.1075 


Compare the brackets, obtained from the 

bracketing procedure, to be used in calculating 

the maximum of the function: (a) without using 

the derivatives, and (b) using the derivatives. 34 
Use these brackets to perform one iteration of 

each of the quadratic and the cubic algorithms. 

Compare the values obtained from the two 

calculations. 


Starting with the bracket | < x < 3, determine an 
approximation to the maximum of the function 


f(x) =x(e* - ee) 


(a) using two iterations of the quadratic 
algorithm, and 

(b) using two iterations of the cubic algorithm. 

(c) How many iterations are required to obtain 
three-figure accuracy? 
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Use the quadratic algorithm to obtain an estimate of 
the value of x that gives the minimum value to the 
largest root of the eigenvalue equation (that is, 


min[A inax()) 


x-aA -1l 0 
1 A 1|=0 
0 -l x-A 


Use the bracket given by x = 1 andx=-1. 


Show that if x, =x, — A and x,; =x, +h then (10.10) 
reduces to 


fi-fs 


xt =x, +1p—t ah _ 

fi- 2ht+h 
(Note: This provides a better hand computation 
method than the Lagrange interpolation approach. 
The best x value is chosen and h is replaced by 
th after each step.) Show that this formula is a 
numerical form of the Newton—Raphson method 
applied to the equation f’(x) = 0. 


An interval AB is divided symmetrically at points C 
and D. If AC/AD = AD/AB show that C divides AB 
in the Golden Ratio a@ = 1/2(3 —/5) = 0.382. ... 
The function f(x) is known to have a maximum in 
the interval a, <x < a,. It is evaluated at the points ,, 
a, and at the golden section points a, = (1 — aa, + Oa, 
and a; = Oa, + (1 — ay. If f(a.) > fla;) then the new 
bracket is taken as a, < x < a,; if f(a) > f(a,) then 
the new bracket is taken as a, < x < a,. The method 
is then repeated. Test the method on the functions 


(a) f(x) =x sin x with bracket 0, 2.5; 
! (in x +202) with bracket 
(1-x) 1 +x 
1:5, 2:5. 


(b) f(x) = 
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10.4.3 


Simple multivariable searches 


As indicated in Section 10.4.1, many multivariable search methods use a sequence of 
single-variable searches to achieve a maximum. The fundamental question is how to 
choose a sensible direction in which to search for the top of the hill with a very limited 
amount of local information. The problem can be visualized when no derivatives are 
available as sitting in a dense fog and trying to get to the top of the hill with only an 
altimeter available. If derivatives are available then the fog has lifted a little, and we 
can now see a few feet around, so that at least we can see which is the uphill direction. 
The criteria for choosing a direction are (a) an easy choice of direction and (b) one that 
gives an efficient climbing method. This is not a simple task, and although the methods 
described in this section are rarely used nowadays, they do provide the basis for more 
advanced methods. Modern methods are difficult to program, not because the basic 
method is difficult but because of the vast amount of remedial action that must be taken 
to prevent the program failing when something goes wrong. The general advice here is 
to understand the basic idea behind a method and then to implement it using a program 
from a reliable software library such as NAG (distributed by Numerical Algorithms 
Group Ltd of Oxford, UK) or a package such as MATLAB. 

Perhaps the most obvious method of choosing a search direction is to use the locally 
steepest direction. For the function /(x,, x5, ..., x,,) this is known to be in the gradient 
direction G = grad f= [Of/0x,, Of/Ox,, ... , Af/Ax,,]. If the derivatives are not available 
then in current practice they are evaluated numerically. 

The gradient direction can be easily proved to give the maximum change. From a 
given point (a,,..., a@,), we proceed in the direction (h,, /,,..., h,,) with given step 
length h so that Aj +h} +...+h2 =’. We then require 


max F'= f(a, +h,,...,a,+h,) —f(a,, ---5 Gn) 
subject to the constraint 
hi+...¢h =P 


The problem is one of Lagrange multipliers, so 


(2) pa oh, Gat 


oh, ~ Oh, 
Thus 
HF _ OF _n9, Ga 
Dh ok 2Ah; (i=1,...,n) 
and hence 
[h,, h,,..., ,] is proportional to ah Boos cae G 
Ox, x, 


The method of steepest ascent (or steepest descent, for minima) then proceeds from the 
point a by choosing the gradient direction G (or —G for minima) for a search direction. 
We therefore need to find 


max{g(d) =f(a + tGi, ay + tG,, sey Ay + tG,,)} (10.13) 
t 
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Figure 10.13 
Steepest-ascent 
algorithm. 


Example 10.16 


Solution 
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read (keyb, a,,..., a,) 
repeat 
{evaluate f(a) and G, = Of/0x,, G, = Of/0x,,... } 
{maximize g(t) in (10.13) by the cubic algorithm 
to give a new pointa<a+t,,,,G} 
until {G = 0} 


Since (10.13) is a function of a single variable ¢, the methods of Section 10.4.1 are 
appropriate, and we should expect the cubic algorithm to be used in the optimization. 
Once the best available point in the search direction has been found, a + t,,., G, the new 
gradient direction is computed and the whole process is repeated. The algorithm is 
fairly straightforward, and is summarized in Figure 10.13. 

The steepest-ascent (or descent) method has the great advantage of being simple and 
secure, but it has the disadvantage of being very slow, particularly near to the optimum. 
It is rarely used nowadays, but does form the basis of the hill climbing methods described 
in Section 10.4.5. 


Find the maximum of the function 
F(X, X2) =-Q - 1) —Q- x) 
by the steepest-ascent method, starting at the point (0, 0). 


It is clear that (1, 1) gives the maximum, but this example is used to illustrate the basic 
method. The gradient is easily calculated from the partial derivatives 


) ) 
2 —a(,-1)'- 20-2), = 20-x) 
1 2 
Cycle 1: At the point (0, 0), f=—1, G= [4, 0], the search direction is x, = 4t, x, = 0, 
and we require 


max{ge(t) = —(4t— 1)*- 1627} 
t 


This can be calculated as ¢,,,, = 0.102 56, so that we can start Cycle 2. 
Cycle 2: The new point is (0.410 25, 0), f= —0.259 and G = [0, 0.8205]. The next 
search is in the direction x, = 0.410 25, x, = 0.8205¢, and we require 


max{g(t) = —0.1210 — (0.410 25 — 0.82057)?} 


This has the obvious solution ty, = 53 so that we can move to Cycle 3. 
Cycle 3: The new point is (0.410 25, 0.410 25), f=—0.1210, and G = [0.8205, 0]. 
The calculation can be continued until G = 0 to the required accuracy. 


There are one or two points to note from this calculation. The function value is steadily 
increasing, which is a good feature of the method, but after the first few iterations the 
method progresses in a large number of very small steps. The successive search directions 
are parallel to the axes, and hence are perpendicular to each other. This perpendicularity 
is just a restatement of the known result that the gradient vector is perpendicular to 
the contours (see Section 3.2.1). In Example 10.16 the function g(f) is written down 
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explicitly for clarity. In practice on a computer this would not be done, since once the 
search direction has been established, x, and x, are known functions of t only, and by 
the chain rule we have 


dg _ afdy,, Afdx,_ Af e , af 
dt ox, dt . Ox, at Pee ie 


Since x, and x, are known, both df/dx, and Of/dx, can be calculated, and G = [G,, Gp] 
is the known search direction; therefore dg/dt is computed without explicitly writing 
down the function g. 

The major criticism of the steepest-ascent method is that it is slow to converge, and 
so the question arises as to how it can be speeded up. One method in use in many 
programs is to use a fixed number of iterations in the line search, provided the function 
is increased. Some experimentation is required on how to implement this idea, but it 
can lead to significant improvement in speed. 

It is well known that Newton—Raphson methods (see Section 9.4.8 of Modern 
Engineering Mathematics) converge very rapidly, so the same basic idea is tried for 
these problems. It is convenient to employ matrix notation, and indeed most multidi- 
mensional optimization methods are written in matrix form. This gives a compact nota- 
tion, and arrays appear naturally in computer languages. 

Taylor’s theorem (see Section 9.4.1 of Modern Engineering Mathematics) can be 
written in matrix form to second order as 


AQ, tM oo 5 Ont My) = fly, 2, FAG + ENG (10.14) 
where 
of Tf Of 
hy Ox, Oxy Ox,OX, 
h=|:|, G=|: |, J=| : ; 
h, Of of af 
Ox, Ox,OX, ax. 


The form (10.14) can be verified by multiplying out the matrices and comparing 
with the standard form of Taylor’s theorem. The vector G is just the gradient vector, 
which is now written in matrix form as a column vector, and J is ann X n symmetric 
matrix of second derivatives called the Hessian (or Jacobian) matrix. 

The Newton method takes (10.14) as an approximation to f, finds the maximum 
(or minimum) of the quadratic approximation, and uses this best value to start the 
cycle again. 

The optimum of (10.14) is given by Af/oh; = 0 (i= 1, 2,..., 1). The first of these 
conditions gives 


0 
o- #11 0 0... O]G+![1 0 ... OWh+thyfo} (0.15) 
1 . 
| 9] 


www.2Ofile.org 


Figure 10.14 Newton 
algorithm. 


Example 10.17 


Solution 
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Noting that, since J is symmetric, for any vectors r and s we have 
rjs=(rjs)'=sJr 

and hence (10.15) can be written as 
O=[1 0 0 ... 0] (G+Jh) 

Similarly for the other components we obtain 


O=[0 1 0 ... 0](G+JA) 


0=[0 0 0... 1](G@+Jh) 

The only way to satisfy this set of equations is to have 
G+Jh=0 

and hence, provided the inverse exists, 
h=-J'G 


The basic algorithm is now very straightforward, as indicated in Figure 10.14. 


repeat 
{a; known, calculate G;, J;} 
{evaluate aj4, = a, — Jj/G;} 
until {sufficient accuracy } 


When Newton’s algorithm of Figure 10.14 converges, it does so very rapidly and 
satisfies our request for a fast method. For a quadratic function it only takes one itera- 
tion so, provided the function looks like a quadratic, the method will be expected to 
converge rapidly. 


Use Newton’s method to find the maximum of 


A=(h+b+ 10)( 227200 as 


+2645) 


Here we have returned to the ‘milk carton’ problem of Example 10.2. We can calculate 


OA _ (2272000 25+5)-(h+b+ 10) 2.272.000 
oh hb Wb 
LL (2222000 ON + 0b 5] +(h+b+ 10) (ee sae 2) 
ab hb b 
aA 2 272.000 


WW =2(b+ 10) 


hb 
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aA 
ob oh 


aA 


ob’ 


Iteration 1 


Anew _ 


Iteration 2 


Qiew = 


Iteration 3 


Anew Pa, 


_ 5, 10x2272000 


| 141.7 _ | 0.6067 
| 61.1] | -0.0552 


—0.0552)| —4.493 
0.0445 | | -98.76 


Ib’ 

ae 2X2 272 000(h+ 10) 
hb 

| 100 _ [-44.92 _ [4.998 2.227 
| 100} S754) 2.227 8.998 
| 100 _| 0.2249 -0.0557]|-44.92) _ [131 
|100} |-0.0557 0.1249] | 375.1 50.6 
[131 _ | -52.3 y= (2421 2.517 
| 50.6” —465.7|’ 2.517 41.75 
131 _ |-0.4407  —0.0266 —52.3| _ 141.7 
| 50.6} |-0.0266 0.0255} | 465.7| 61.1 
[141.7 _ | -4.493 y = {71858 2.303 
| 61.1 , ~ 1-98.76 |’ | 2303: 25,33 


139 


ea 


The iterations converge very rapidly; h = 139, b = 65 is not far from the solution, and 
gives A = 827 cm’. 


Since the Newton method involves matrices MATLAB proves to be a very suitable 
package to perform the manipulations. The simple instructions 

Z= (LOO: LOW] 

la, Gp ad | Saeco (4 (iL) ,2 (A) ) , a=2=I We” 


with the last line repeated, gives the successive iterations of the example. The 
following listing gives the m-file newton.m that is used for Example 10.17. 


function [a,agrad,ajac] =newton(h,b) 


tl=h+b+10;t2=2272000/ (h*b) +2*b+5; 


asic il 


eiopeeiel (1) Se Se LT AO // (lax 2195) & 
elopareNel((2)) 1 eee tht (2 NOOO) / (atatloye 2) )) 4524 )) 2 
aqacdl, L)\=2° (os ilO)) 227 2000/7 (ms) 15)) 5 


ajac 
ajac 


(1,2) 
(273) Aa (nA) 227 2000 / (a3) 
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S2E O22 7 2000/ (la 2 ley" 2) paTae (2 , IL) eajere (i, 2) F 


Figure 10.15 
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Unfortunately Newton’s method is very unreliable, particularly for higher-dimensional 
problems, unless the starting value is close to the maximum. The reason for this is fairly 
clear. The analysis given only uses the necessary condition for a maximum, but it would 
apply equally well to a minimum or saddle point. In multidimensional problems saddle 
points are abundant, and the usual failure of the Newton method is that it proceeds 
towards a distant saddle point and then diverges. 

Applying the method to the Rosenbrock function, called the ‘banana’ function in 


MATLAB, 


F(x, y) = 100(37 — xy’ + (1 — xy 


the unreliability, but rapid convergence, 1s illustrated in Figure 10.15. 


Behaviour of Newton’s Starting point Iterations to convergence Final point Comments 

method for the - 

Rosenbrock function (-1.9, 1) 1 (1, 1) eee 

for various starting (-1.9, 0.9) 6 (1, 1) minimum 

points. (—1.9, 0.5) 6 d,-1) minimum 
(-1.9, 0) run 1 - - aborts, J is singular 
(-1.9, 0) run 2 1 (1/101, 0) saddle point 
(-1.9, -0.5) 7 (1, 1) minimum 
(-1.9, -1) 1 (1, -1) minimum 

10.4.4 Exercises 
35 Follow the first two complete cycles in the 37 Use the steepest-ascent method to find the 


36 


steepest-descent algorithm for finding the 
minimum of the function 


_ r| 1 il r| 1 =1 
F(x, X%2) =x 0 rad “1 > |* 


1 
starting at a= Hl : 


Show that the function 


f(%, ¥) = Ax + ypP + (H—yP + 3x4 2y 


has a minimum at the point (-2, -3). 


Starting at the point (0, 0), use one iteration 
of the steepest-descent algorithm to determine 
an approximation to the minimum point. 
Show that one iteration of Newton’s method 
yields the minimum point from any starting 
point. 
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39 


maximum of the function 
fy, 2)=-@—y+z) — (2x 42-2" -@—-1P 
starting from (2, 2, 2). 


Minimize the function 


fix, y,2)=@—y +2) +(Qxtz-2P+(@-1P 


by Newton’s method. starting at (2, 2, 2). 


A new link road is to be constructed from a city centre 
to an existing road. In suitable coordinates and units 
the city centre is at (0, 0) and the existing road has 
equation y = 11 — 2x. The cost of construction is 
proportional to the length of road, but it is twice as 
expensive to construct the road in the urban region 
|x |S 1 compared with outside the region. The link 
road consists of two straight sections, inside and 
outside the urban region. Find the equations of the 
two sections that minimize the overall cost. 
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10.4.5 


Advanced multivariable searches 


To overcome the problems of evaluating second derivatives, which are rarely available, 
and of the unreliability of the Newton method, but to use its speed of convergence, 
several methods were produced in the early 1960s. Two have survived and are now the 
methods currently available in most program libraries. Conjugate-gradient methods 
give one approach, but these will not be described here. We shall look at a method due 
to Davidon, commonly called DFP (after Davidon, Fletcher and Powell, who developed 
the method) or quasi-Newton methods. There is a whole class of such methods, but 
only one will be studied. 

The basic idea of the DFP method is to look for the minimum of the function 


f(%,,---.,%,) with gradient given by the column vector G=[df/ox, of/ox, ... oflox,]' 


by iterating with a matrix H,, which will be updated at each iteration, so that 

a;,, =a, — AMG, (10.16) 
The reliable but slow steepest-descent method chooses H, = I, the unit matrix, and A=1.,,..5 
while the less reliable but fast Newton method chooses H, = J ;' and A= 1. Thus 
the idea is to compute a sequence of H, so that H, = land H, > J“! as the minimum is 
approached. The basic analysis required to implement this scheme is quite difficult and 
beyond the scope of this book, so only the briefest of outlines will be given. For a 
quadratic function 

f=ctxiGt Lyi) Xx; 
at two successive points the gradient is given by 

G,=G+)Jx, 

Gi = Gt J Xi 
so, subtracting, 

Gi — Gi =) Xin — X) 
Writing h; = x;,; — x; and y; = G,,,-— G, and working on the assumption that HJ = I 
since we require H, > J ~', we obtain 

Hy, = HJ h,=h; (10.17) 


It is found to be impossible to satisfy (10.17) unless an exact solution is known, so the 
next best thing is to satisfy (10.17) one step behind as 


H... 9 = 1; (10.18) 
There is a whole class of matrices that satisfy the key equation (10.18) but only the 
original Davidon matrix (and still one of the best) is quoted here, namely 
H.y.y:H, hh; 

y iH; hiy; 
It can be shown that for a quadratic function this sequence of Hs produces J “ in 
n iterations, where n is the dimension of the problem. The basic algorithm is described 
in Figure 10.16 for the minimum of a general function f(x,, ... , x,,) with gradient 
G=[odflox, ... dfldax,]'. 
This method was a major breakthrough in the early 1960s, and is still one of the best 
and most reliable available. Proofs of convergence and computational experience are 


H,,, =H, - (10.19) 
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Figure 10.16 
DFP algorithm for 
the minimum of 


S(%, %, san’ 5 Xy,)s 


Example 10.18 


Solution 


Iteration 1 
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read {initial values xo, Hy = I} 
{calculate the gradient G, and fy} 
repeat 
{Find min f(x, — AH,G,), by the cubic algorithm} 
{Put Xi.) =X; — A minH{G;} 
{Calculate f,,,, G;,, and hence h; = x,,, — x, 
and y; = G,,, — G;} 
{Update H; to H;,, by (10.19)} 
until {sufficient accuracy} 


available in advanced texts on optimization. To repeat a word of warning: these pro- 
grams are very long and tedious to write because of the large amount of checking and 
remedial work that has to be inserted to prevent the program stopping. Such programs 
are available in software libraries, and these should be used. 


Use the DFP method to find the minimum of 
fx, y)=xi + y+ (2x + y—5P 


starting at (0, 0). 
Note that only the first derivatives 


G= ? 
4y +2(2x+y-5) 


4x° + 4(2x+y- i 


are required in this method. 


and we search in the direction 


sf a). i 


for a minimum of /; that is, we compute 


min{(20A)* + (104)* + (50-5) } 
a 


The cubic algorithm gives A pin = 0.064 14, so 


1.2828 1.2706 
a, = . G, = 
0.6414 —2.5308 


1.2828 —21.2706 
hy= > Sor 
0.6414 7.4692 
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Thus H is calculated from (10.19) as 


l= 


_ | 0.1611 —0.2870 
—0.2870 =0.9031 


2 1.282 
Iteration 2 a,= , 


. G= 1.2706 He 0.1611 —0.2870 
0.6414 


= , f= 6.092 
~2.5308 ~0.2870 0.9031 


We now search in the direction 
-_— 1.2828] _ A 0.1611 —0.2870 1.2706 
0.6414 —0.2870 0.9031] |—2.5308 
__ | 1.2828 - 0.9309 
0.6414 + 2.6501A 


and the cubic algorithm gives 1 ,,;,, = 0.272. We can now compute the next point and its 
gradient as 


ne 1.1782 oe —0.2761 | f=56101 
0.9390 —0.0969 


The new Davidon matrix is calculated from (10.19) as 


H. = 0.0597 —0.0050 
—0.0050 = 0.1191 


Iteration 3 The next iteration gives 


: —0.021 
tig | TABTE) gee | -O0T22). yg | ORE ONO). £25. 6088 
0.9452 0.0192 —0.0216 0.1019 
The iterations continue until convergence at x = 1.1886 and y = 0.9434. 


In current practice the steepest decent method is rarely used because it is too slow. 
Newton’s method requires a matrix of second derivatives, which are not usually available, 
and, although fast, it is too unreliable as illustrated in the previous section, particularly 
in higher dimensional problems. The quasi-Newton methods, however, are widely avail- 
able in most libraries and packages. They are reliable, very competitive and compare 

favourably with other well used methods, such as conjugate gradients. In MATLAB the 
optimization routine fminunc, DFP and steepest descent are available, but a variant 
BFGS (Broyden, Fletcher, Goldfarb, Shanno — see Exercise 40) is the default method. 
In the Optimization Toolbox of MATLAB the ‘unconstrained nonlinear’ option contains 
a demo of the three methods on the Rosenbrock function (it was also used to illustrate 
Newton’s method in the previous section). 


F(%, y) = 100(y? — x)? + (1 — xy 
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There are some first class graphics showing the progress of the method and the con- 
vergence found from the starting point (—1.9, 1) is 


BFGS 34 iterations 50 function evaluations 
DFP 40 iterations 64 function evaluations 


Steepest descent exceeds limit 250 function evaluations 


This performance is typical of the methods. The complication of the methods and 
the intimate relation with computers illustrates the need for packages to perform the 
extensive arithmetic. For the milk carton problem, Example 10.2, the function infor- 
mation is put in the M-file milk.m 


‘ewNale Loin [[3E , © ||anaL LIK (Se) 
= (seats (2) PLO) = (22 7 2000 / (se (AL) 32 (2) ) see ess (2) ae5)) 2 
if nargout>1 
G (I )SH(227 2000/7 (se (1) se (2) see se (2) sed) = (se (AL) see (2 se) 
OPT AO) (Se (1) 2x22.) )) 5 
(2 VS=(22 72000 / (se (AL) as (2) ake se (2) eS) )) se (Se (AL) ase (2 SILO) 
(S227 2000 / (2 (1) 22(2) 2 )ar2) 9 
end 


The instructions 


x0=[100;100]; 
options=optimset (’GradObj’,’on’,’display’,’iter’); 
[ee Ewell || Satin archae: (Cima I<, <0), CFOIE LCiaKss )) 


produce the minimum of 827 cm? with A = 138.6 mm and b= 65.7 mm in six iterations. 


A major development since the 1970s has been in devising modifications that avoid 
the line searches. It was found that the latter were very time-consuming, so there was 
great pressure to avoid them. The searches were replaced by one or more steps in the 
search direction until the function has been reduced ‘sufficiently’ and then the matrix 
H is updated. This not only reduces the number of function evaluations, which is nor- 
mally the most expensive part of the routine, but it is found to reduce the number of 
iterations required. The BFGS variant is found to be very suitable for this approach and 
it is used in the MATLAB implementation. What is meant by ‘sufficiently’ requires 
careful consideration and a discussion can be found in R. Fletcher, Practical Methods 
of Optimization, Volume 1, Unconstrained Optimization, Wiley, 1980. 

The early development of numerical optimization algorithms led to very distinct methods 
for cases when derivatives were or were not available. As computers developed in 
speed, this difference became less necessary, since derivatives could be calculated very 
rapidly by a numerical method. Options are now built into packages and library routines 
which use derivatives when supplied, or use a numerical approximation if the deriva- 
tives are not supplied. 

Adapting methods to deal with constraints is of intense interest in practice, and has 
been a major thrust in the subject. For fully nonlinear problems with nonlinear con- 
straints the practical difficulties are very severe, but, robust programs are now available 
in most libraries and packages. 
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10.4.6 


Least squares 
When minimization problems are in the form of least squares 
F= fi, X25 +++ 5 X,) +f 3%, Xa, +665 X,) +... + fa(%1, X25 +++ 5 X,) 


then there is a slightly different and potentially more efficient technique that exploits 
the specific structure of the problem. In matrix form F becomes 


fi 
F=Lfihi-.-fall? 
Tn 
Each of the functions is expanded by Taylor’s theorem, in matrix form, to first-order 
about x = a, for example putting x, =a, +h,,x,=a,+h,,... for, 
hy 
Of, AF; Of; | |A2 
= (X1,%.,...,X%,) = 05.45 08 Og) SS ee 
fi = G1, % J=fi (Ai, a ) ae 2, || : 
hy 


The partial derivatives are evaluated at x = a. Repeating for all the m functions gives in 
matrix form 


HO of 


ni fn x =a nn Hn fm i. 


| Ox, Ax, Beg 


or in a more compact notation 
f(x) =f(a)+Jh (10.20) 


The minimum of F requires 


ti 

o=2F i 2|%H BA Al |h 
Ox |ax, ax, &, ||: 
Fo 


Repeating this differentiation for each of the variables and putting the equations into 
matrix form 0 =J 'f(x) and using (10.20) gives 


0=J 'f(x)=J (fla) +Jh) 
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Solution 


Example 10.20 
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Hence we can now compute h as 


h=-J JVI 'S@ 


provided, of course, that the inverse exists. This is the same result that was quoted in 
Section 1.8.3 on singular value decomposition. To compute the minimum the result is 
iterated as 


a,=4,-O0) VS (10.21) 


Find the minimum of the function, starting at (0, 0), 


F=(xt+y—1P+(e—yt 1) + (2x-y? 


xt+y-1 1 1 
f=l|x-ytl| J =]1 -1 
2x-y 2 -1 


so at the start point 


-1 
1 


From (10.21) the new value of a is (2/7, 6/7) with F = 2/7. Because all the functions are 
linear the minimum is obtained in one iteration. 


MATLAB has a built-in procedure to solve this type of problem; the following 
instructions obtains the same result quickly 
(CSL, dei, apa, = peealilgaLpullies & = Gz = 
x=lsqlin(c,d,[ ],[ ]) % Can use linear equality and 
inequality constraints, [ ] indicates empty 


The experimental data 


Estimate the values of a and b and compare. 
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Solution 


Figure 10.17 
Fit of the function in 
Example 10.20. 


The minimization of the least squares function 


4 
x, 
Fi : b) = — wird 
Me?) Llu 1 +x, 
should give good estimates of a and b. Note that the origin automatically satisfies the 
function. Take 


Lf - aX, | = 7 
© 1 +bX, ah ee 
f- aX, 1+bX, (1 +bX,) 
fs ba andj = |_=*%2 aX; 
fo 1+bX, (1 +bX,)’ 
1 +bX; ae a oe oe 
aXe fo, 
fry +bX, : = 


Starting at a=1 and b=0 .5 the iteration (10.21) was written in MATLAB and quickly 
gives a converged value of a=1.0779 and b=0.4764. However starting from (1, 1) 
the method quickly diverges. It is important that a good starting guess is known. The 
code 


ezplot (‘/1.0778*x/ (1+0.4764*x)’,[0,10]), hold on 
PLOEO,/059 7% ** , 1.270 28, 88, 2p L15y ee? 7 Spb, oe 10,1 .89;.°** ) 


produces Figure 10.17, which gives an illustration of how good the fit is. Note that least 
squares often gives a better fit to experimental data than interpolation methods, such as 
splines, since rogue points are not dominant in the method. 


1.0778x/(1 + 0.4764x) 
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MATLAB has built-in procedures to deal with exactly this type of curve fitting. The 
following instructions obtain the same result. 


He-10 1.2 2 5 1Olpwe=(0 O.80 1.15 1.55 1.89) & aaseccis 


the data 


gg=@ (b, xd)b(1)*xd./ (1+b(2)*xd); % sets up the test function 
x=lsqcurvefit(gg, [1 1],xd,yd) % returns the values 


x = 1.0779 0.4764 


A much wider range of start values can be used since 1sqcurvefit can access a 
variety of methods of solution and contains numerous safeguards. 


10.4.7. Exercises 


Minimize the following functions by the DFP 
method, completing two cycles: 


(a) f(x, y)=(«—yy + 4x — 1)’, starting at (2, 2); 
(b) fa, y,z)=(e-y tz)? 4+ (Qx4+72-27 47, 
starting at (0, 0, 0). 


Show that the updating formulas 


(h, - H.y))(hi- Hy)" 


(i) Hj. =Hj;+ = rank (1) 
(h;- Hiy,) y; 

and 

T t 
(ii) H,,, =H, +] 1 +2 A = 

hiy; hiy; 

T T 
- (os Met (BFGS) 
iVi 


satisfy (10.18). Follow the DFP method through for 
two cycles, but using these updates for H, on the 
functions 


(a) x* + 2y’, starting at (1, 2); 
(b) x°+(x-y+ 1) +y7z’, starting at 
(0.5, 0.5, 0.5). 


Show that the update formula (with suffixes 
suppressed) 


H’=H+vp' — Huq’ 
satisfies the basic quasi-Newton equation (10.18) 


H’u=v 
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where p and q are vectors satisfying 
pu=l, qu=1 


but are otherwise arbitrary. 
By making a suitable choice of a, Band a’, B’ in 
the expressions 


p= av + BHu 
q=a'v + B’Hu 
show that the Davidon formula (10.19) and formula 


(1) in Exercise 41 can be obtained. 


An alternative algorithm for finding the minimum 
of a function, f(x) with gradient g, of several 
variables is due to Fletcher and Reeves. Starting 

at the point xp, the first search direction is chosen as 
Po = —&-. Successive search directions are given by 


T 
&i $i 
P= -8:+ — Pi 
8i-1 8-1 
and successive points satisfy 
X =X t Api 


where A,_, is chosen to minimize the function f(x) 
in this search direction. 

Apply the method to the functions 
(a) f= $x? +’) and 


(b) f=(x-y+ 1) 4+x’*y? + (z- 17 
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Ree taredialccvalaccarls) irl Chemical processing plant 


A chemical processing plant consists of a main processing unit and two recovery units. 
Chemicals A and B are fed into the plant, and produce a maximum output of 100t day”! 
of material C. The effluent stream is rich in chemical B, which can be recovered from the 
primary and secondary recovery units. The total recovered, when at full throughput, is 
10 t day’ of pure B, and it is fed back into the incoming stream of chemical B. The process 
is illustrated in Figure 10.18; the numbers in parentheses indicate the maximum flow (in 
t day”') that can be sent down the pipes, and the x; indicate the actual flow (in t day'). 

The chemistry of the process implies that the chemicals must be mixed in given 
ratios. For the present system it is found that 


Xx Sl, x%y= 3:5, wing 321, xXx, = 5:3 
HeNeHIi2,. Kexy = O11, AzeixXyg='6:5 


must be maintained for any flow through the system. Chemical A costs £100t™, chemical B 
costs £120t™' and chemical C sells for £220t'. The running costs are as follows: 


Variable costs Fixed costs 
Process unit £70t" of product £500 day” 
Primary recovery £30t" of input £200 day” 
Secondary recovery £40t of input £100 day” 
Disposal of waste £30t7 
Indirect cost £400 day” 


It is required to find the most profitable operating policy that can be achieved. 
The profit can be written down for a day’s production as 
z =—(fixed costs) + (profit from sale of C) — (costs of chemicals A and B) 
— (process unit costs) — (primary unit costs) — (secondary unit costs) 
— (waste product costs) 
= —1200 + (220x,) — (100x, + 120x,) 


(70x,4) — (30x5) — (40x7) 
— 30x49 


z=—1200— 100x, — 120x, + 150x, — 30x; — 40x, — 30x19 


Figure 10.18 x, (60) 4 (100) 
Schematic diagram je x (60) 
of a chemical B— ae 
processing plant. x, (50) |K 
The numbers in 
Xy (10) A 
parentheses give the Primary 
maximum flow. J on B recovery 
bi) 
ve 2) B recovery To waste 
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Modification to 


processing plant. 
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The constraints on the flow, given by the maximum throughput, are 
x,<60, x, <50, x,<60, x,< 100, x,<20, x,<8, x, < 12, 
XyS2, x, 10, x, < 10 

The constraints on the chemistry given by the fixed ratios can be written in a convenient 

form as 
x,-x,;=0, 5x,-3x,=0, x,-—3x,=0, 3x,—5x,=0, 2x; —-5x,=0, 
X,—6x,=0, 5x,- 6x,,=0 

Finally, at the junctions J and K, continuity (what flows in equals what flows out) gives 
Xgp+%y—X=O0;, x,+x,—x,=0 


The problem thus has 10 variables, 10 inequality constraints and 9 equality constraints. 
The choice is whether to use the equality constraints to eliminate some of the variables 
or just to treat the 19 constraints directly by LP. The equations are sufficiently simple 
to solve for the variables as 


2 1 
TsX1r X7 = FX, Xe 


5 _ 5 1 _ 
X= GX XZ =X, Hg ZX, AS =AZX, XG 


x= 2x1, X= tx, z= 27x, — 1200 
Thus x, must be as large as possible, that is, at the value 60, giving a maximum profit 
of £420 day’. We must check that all the constraints are satisfied, and indeed this is the 
case. It is easily seen that each variable reaches its maximum possible value indicated 
in Figure 10.18. 

When we look at variations on the problem, it becomes less clear whether to elimin- 
ate or just to use LP directly on the modified equations. For instance a very sensible 
question is whether it is worth using the primary or secondary recovery units. We can 
consider this question by allowing a portion to go to waste (at the same cost given 
previously), as indicated in Figure 10.19. 

We add to the previous continuity equations similar equations for the junctions L 
and H: 

Xs — Xj, —Xy.=0 

X7—Xy3 —X4=0 
The inputs to the primary and secondary units are now x,, and x,,, so we need to modify 
the fixed ratio chemical constraint as follows: 

replace 3x;—5x, =0 by 3x,,-5x, =0 


replace 2x;—5x, =0 by 2x,,—-5x; =0 


X;, To waste 


X13 To waste 


Xi9 To waste 
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replace x,-—6x, =0 by x ,-6x, =0 
replace 5x,-—6x,=0 by 5x,4—- 6x,)=0 


In the cost function x; is replaced by x,,, and x, by x,,, and the additional waste costs 
(-—30x,, — 30x,;) must be added: 


z= -1200 + 150x, — 100x, — 120x, 
30x, — 30x,, — 40x,, — 30x,3 — 30Xj9 


We now have three free variables, so we shall certainly need an LP approach. An LP 
package was used to obtain the solution 


x, =57.69, x, =50, x; =57.69, x, = 96.15 
x5 =19.23, x6 = 7.69, x, =11.54, x = 0 
Xy = 7.69; x= 9, xii = 0, Xi = 19.23 
X,3=11.54, x,= 0 


and the profit is £530.77 day~’. It can be seen that x,, = 0, so nothing is sent to waste 
before the primary recovery unit; but x,, = 0, so that all the material from the primary 
to the secondary goes to waste, and the secondary unit is bypassed. The effect of this 
strategy is to increase the profit by about 20%. 

There are many other variations that can be considered for this model. For instance, 
pumps often go wrong, so it is important to investigate what happens if the maximum 
flows are reduced or even cut completely. Once the basic program has been set up, such 
variations are quite straightforward to implement. 


10.6 Engineering application: ite diteaie 


A heating fin is of the shape indicated in Figure 10.20, where the wall temperature is 
T,, the ambient temperature is 7) and within the fin the value T = 7(x) is assumed to 
depend only on x and to be independent of y and z. Heat is transferred by conduction 
along the fin, which has thermal conductivity k, and heat is transferred to the outside 
according to Newton’s law of cooling, with surface heat-transfer coefficient 4. Consider- 
ing an area of the fin of unit width in the z direction and height 2y, the heat transferred 
by conduction in the x direction is k2y d7/dx. The net transfer through the element 
illustrated is (d/dx)(k2y d7/dx). The total heat lost through a surface element of unit width 
in the z direction and length As = [1 + (dy/dx)"]'” Ax along the surface is h(T— T,)As. Since 
there are two surfaces, we can write the heat-transfer equation as 


a} dor es . dy)’ 
mac air r0| 2A(T ro) +()] 


Provided that dy/dx is not too large, (dy/dx)’ can be neglected, giving 


1/2 


a} dor aire 
tlre r0| rd To) (10.22) 
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Figure 10.20 
Heating fin. 


Ambient temperature Ty 


The mass of the fin is given, so that its cross-sectional area is known, and hence 
| ydx=}A (10.23) 
Finally, we wish to maximize the heat transfer, so we require the maximum of 


l= 24 | (T- Ty) dx (10.24) 
0 
over all possible functions y(x). 

The problem involves choosing a function (x) that satisfies (10.23) and then solving 
(10.22) for T— T). This is then substituted into (10.24) and the integral evaluated. 
Out of all possible such functions y, we choose the one that maximizes J. The scheme 
outlined is extremely difficult and belongs to a class called variational problems. 
An alternative approximate method must be sought. The assumption made is that the 
temperature falls linearly with x as 


T-—T,=(T, — T))( — ax) 
We also assume that y = 0 at x =a. We thus have two free parameters, @ and a, which 


we can use to give an approximate solution. Given JT — 7), y can be computed from 
(10.22) as 


y= A (a taa" ~ x+ Lox’) 


It can be seen that our basic assumption implies that y is quadratic in x. To satisfy the 
area constraint (10.23), a simple integral is performed to give 


h 

LAs iat -1@a) 

which gives a relation between a@ and a. The function / is now integrated as 
T=2h(T, - r» | (1 - ax) dx 


0 
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or 


I 1 2 
——— =q- 
2h(T, - Ty) 2 


Thus the very difficult problem has been reduced to a Lagrange multiplier problem of 


maximizing 
f=a- 5 aa 
subject to 
2 
g=0= ( -1q°- s*) (10.25) 


where S* = kA/2h. The Lagrange multiplier analysis gives the equations 


Af 48) | - ga-a(4-a") =0 
Oa a 


2 
Af-Ag) __ 1244 a -0 


0a 2 20 


Hence 
A=oe 
1-2aa+ 0a’ =0 


so that aa = 1. Substituting back into the constraint (10.25) gives 


1/3 
cid =1a’°, or a= Ga 


and therefore 
(T- 1) =(T,- To)(1-2) 
so that 
y_ha (1 7 xy 
a 2k a 
Thus, given the physical parameters hk, h and A, the ‘best’ shape can be derived. 
This model shows how a very difficult mathematical problem in optimization can be 


reduced to a much more straightforward one by an appropriate choice of test functions 
for T — Ty. 
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10.7 Review exercises (1-26) 


i 


Use the simplex method to find the maximum of 
the function 


F= 12x, + 8x, 
subject to the constraints 
Xj + xX, = 350 
2%, + x, = 600 
x, + 3x, = 900 
X4, X%, = 0 


Check your results with a graphical solution. 


A manufacturer makes three types of sailboard 
and is trying to decide how many of each to 

make in a given week. There are 400h of labour 
available, and the three types of sailboard require 
respectively 10, 20 and 30h of labour to construct. 
A shortage of fibreglass and of resin coating is 
anticipated. The quantities required by each type 
of sailboard are as follows: 


Total 


Type 1 Type2 Type3_ supplies 5 


Fibreglass 

(kg) 5 10 25 290 
Resin 

coating 


(litres) 3 2 1 72 6 


If the profits on types 1, 2 and 3 are £10, £15 and 
£25 respectively, how many of each type should the 
manufacturer make to maximize profit? 


A motor manufacturer makes a ‘standard’, a ‘super’ 
and a ‘deluxe’ version of a particular model of car. 
It is found that each week two of the materials 

are in limited supply: that of chromium trim 
being limited to 1600 m per week and that of 
soundproofing material to 1500 m* per week. The 
quantity of each of these materials required by a car 8 
of each type is as follows: 


Standard Super Deluxe 
Chromium trim (m) 10 20 30 
Soundproofing (m*) 10 15 20 


All other materials are in unlimited supply. 
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The manufacturer knows that any number of 
standard models can be sold, but it is estimated 
that the combined market for the super and 
deluxe versions is limited to 50 models per week. 
In addition there is a contractual obligation to 
supply a total of 70 cars (of any type) each week. 

The profits on a standard, super and deluxe 
model are £100, £300 and £400 respectively. 
Assuming that the facilities to manufacture any 
number of cars are available, how many of each 
model should be made to maximize the weekly 
profit, and what is that profit? 


A poor student lives on bread and cheese. 

The bread contains 1000 calories and 25 g 
protein in each kilogram, and the cheese has 
2000 calories and 100 g of protein per kg. To 
maintain a good diet, the student requires at 
least 3000 calories and 100 g of protein per day. 
Bread costs 60p per kg and cheese 180p per kg. 
Find the minimum cost of bread and cheese 
needed per day to maintain the diet. 


Use Lagrange multipliers to find the maximum 
and minimum distances from the origin to the 
point P lying on the curve 


x-xyty=1 


A solid body of volume V and surface area S is 
formed by joining together two cubes of different 
sizes so that every point on one side of the 
smaller cube is in contact with the larger cube. 
If S=7 m”, find the maximum and/or minimum 
values of V for which both cubes have non-zero 
volumes. 


Find the maximum distance from the point 
(1, 0, 0) to the surface represented by 


x+y? +z=8 


Find the local extrema of the function 
FQ, y, Z) =x + 2y + 3z 
subject to the constraint 
e+y+77=14 
Obtain also the global maximum and minimum 
values of F in the region 


x20, y20, 220, x+y? 4+277?< 14 
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A triangle with sides a, b, c has given perimeter 
2s. Recall that the area of the triangle, A, is given 
by the formula 


A’ =s(s — als — b\(s —c) 


(i) Ifa is given, use Lagrange multipliers to find 
the values of b and c that make the area a 
maximum. 

(11) Ifa, b, c are unrestricted use Lagrange 
multipliers to find the values that make 
the area a maximum. 


A nuclear reactor is in the form of a circular 
cylinder of radius r and height h. According to 
the theory of nuclear diffusion, the restriction 


2 2 
eG) 
r h 
applies, where a and b are constants. Use Lagrange 


multipliers to find the values of r and / that make 
the volume of the reactor a maximum. 


According to lubrication theory, the lift on a pad 
bearing, where fluid flows in the narrow gap 
between a pad and a fixed piece of machinery, 

is given by 


F=A 


: [ine- 2&4) 
(k-1) k+1 


where A is a constant, k=h,/h, > 1 and h, and h, 
are the gap widths at the front and back of the 
pad. Find the value of & that makes F a maximum 
by using the bracket/quadratic approximation 
technique. 


A cylindrical can of radius R (cm) and height 
H (cm) is to be made with volume 1000 cm’. 
The cost of making the can is proportional to 


(amount of metal) x (machine factor) 


where the amount of metal is proportional to the 
surface area of the can (including the two ends) 
and the machine factor is given by 1 + [1 — (H/4R)]’ 
and reflects the difficulty of machining the can. 
Show that the cost is 


2( 1000 + 2k) 1+(1 , L990"; 
R AnR 


Find a bracket for R and use the quadratic algorithm 
to estimate the radius that minimizes the cost. 


ies 


14 


ILS) 


16 


Use the quadratic approximation method to 
obtain a first estimate of the minimum of the 
function 


f@)=1-t+0 
where ¢ is the non-negative root of 
?+ix—(1—-x?)=0 


Start with the interval 0 < x < 1 and note that 
for x = 0.5, t= 0.6514 and f = 0.7729. 


In Figure 10.21 the disc rotates at a constant 
angular velocity, so @= at. The subsequent 
movement of the slider P gives x = x(t). If 
L/a =A show that the velocity, v, of the 
slider is given by 


LY —~sing|1 + 288 _ 
aa \(4 -sin’@) 
Use the bracket and quadratic approximation 


technique to evaluate the maximum and minimum 
velocities of the slider in the case A = 3. 


Figure 10.21 Disc and slider in Review 
exercise 14. 


A trucking company estimates that the cost of 
running a truck is 


1 
0.02(20" + # | 
10 


pounds per mile at a constant speed v. The driver 
earns £5 per hour. Find the cost for a journey 

of D miles. What speed is recommended to 
minimize the cost? 


Use (a) the steepest-descent method, (b) the 
Newton method and (c) the DFP method to 
find the position of the minimum of the 
function 


fe, yy=3x7 + @—yh + Loetyt I 


starting at (0, 0). Perform two cycles of each 
method, and compare your results. 


www.2Ofile.org 


18 


A compound pendulum consists of a rectangular 
lamina with a heavy particle embedded in it, as 
illustrated in Figure 10.22. For small oscillations 
about the equilibrium, 6= q, and putting 9= a+ €, 
the equation of motion is 


Ee 
é=-1re 


Figure 10.22 Compound pendulum in Review 
exercise 17. 


where the period of the oscillations LU is given, 
after a substantial calculation, by 


ape _ VAY) +X") 
2 2 2 
Bae ce eas) 
with 
X=x/la, Y=yla, k=bla, A=m/M 


Explore this expression for maximum and 
minimum values in the region |X| = k, Y S 2; 
take the case A=} andk=+. 
A method called Partan uses the notation D® for 
the gradient of fevaluated at x =x. The iteration 
scheme for evaluating the minimum of fusing 


Partan is 
x =x — py DO 
a= uD” 


(23503 


xd = 2 a, Adz” a xe) 


where j1; and A are chosen by optimum line searches. 
Sketch the progress of this method up to the point 
x” for a scalar function of two variables f(x,, x2). 
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Illustrate the use of Partan and the method 
of steepest descent on the quadratic function 
f= = Xp) Cal Dy 


starting at (0, 0). 


The Newton method described in Section 10.4.3 
often fails to converge. One way of overcoming 
this problem is to restrict the step length at 

each iteration. Given that x = a+ h, this can be 
implemented by constraining / to have length L, 
where 


ips Ie 


Use a Lagrange multiplier A to show that the 
result gives 


Xnew = Xold — 0) oF My'G 


The algorithm is then implemented by 
successively using A= 0, 1, 10, 100,... 
until a reduction in the function fis 
obtained. 

Starting atx=[1 1)", perform one 
complete step of the modified algorithm on 
the function 


far +y? —x’y 
Use the method developed in Section 10.4.6 to 
iterate to the minimum of the functions 
(a) F=(x-y’+t(e+y+t 1%, starting at 
sa Oy =, 
iW x “ 
(b) F= (=) + (>) , Starting at 
x+y ee ay 
x=l1,y=1. 
It is known from experience that a curve of 
the form 
y=I1K(a+ bx) 


should give a good fit to experimental data in the 
form of a set of points 


(Vi) (@=1,2,...,p) 


It is required to calculate a and b by a best 
least-squares fit, and thus to minimize 


In 2 


i=1 
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22 


23 


24 


Use the least-squares algorithm described in 


Example 10.20 to fit the function to the data points 


(0,1) (1,0.6), (2,0.3), (3, 0.2) 


A quadratic function f(x), x%.,...,%,) witha 
unique minimum is given in matrix form as 
a Teeveieeed 
f=ctbx+5x'Ax 
Show that a search in the direction 
x=atdd 
produces the minimum at 
T 
Weel b+Aa) d 
atid) 
d' Ad 


Complete two complete cycles of the steepest- 
descent algorithm for the function 


f@, y) = GQ =x + &=yy 


starting at x = 0, y= 0. Use Review exercise 22 
for the minimization in the search directions. 

Show that the minimum is obtained in a 
single iteration of the Newton method. 


(Harder) It is required to solve the differential 
equation 


ay eye 


with the boundary conditions y(0) = 1 and 
y(1) = 3, by a shooting method. The equation 
is solved for the initial conditions 


yO=1, yO=a 


28) 


cE] 


26 


by any suitable method (for example, by a 
Runge-Kutta method). With this solution, 
calculate 


F(a) = y(1) 


and then try to drive F to the value 3 by 
minimization of 


ea) — 3); 


In this example illustrate the method by using 
the exact solution 


y=(1+b)e"—b 


for the forward integration. 


(An extended problem) In the chemical 
processing plant model in Section 10.5 
consider the profits when 


(a) the primary pump is faulty and the 
constraint x; < 12 is imposed; 

(b) the waste pump between primary and 
secondary fails so that x,, = 0 (see 
Figure 10.19). 


(An extended problem) Extend the heating 
fin analysis in Section 10.6, using a higher 
approximation to the temperature: 


T—T) =(T, — T))( — ax — Bx*) 


Compare the shape of the fin with that given 
in the text, and compute the heat transferred 
in each case. 


www.2Ofile.org 


Chapter 11 


Aalalal 


xs I 2 


and Statistics 


Contents 

Introduction 

Review of basic probability theory 

Estimating parameters 

Joint distributions and correlation 

Regression 

Goodness-of-fit tests 

Moment generating functions 

Engineering application: analysis of engine performance data 
Engineering application: statistical quality control 
Poisson processes and the theory of queues 
Bayes’ theorem and its applications 


Review exercises (1-10) 


www.2Ofile.org 


906 APPLIED PROBABILITY AND STATISTICS 


11.1 


11.2 


Introduction 


Applications of probability and statistics in engineering are very far-reaching. Data from 
experiments have to be analysed and conclusions drawn, decisions have to be made, 
production and distribution have to be organized and monitored, and quality has to be 
controlled. In all of these activities probability and statistics have at least a supporting 
role to play, and sometimes a central one. 

The distinction between applied probability and statistics is blurred, but essentially it 
is this: applied probability is about mathematical modelling of a situation that involves 
random uncertainty, whereas statistics is the business of handling data and drawing 
conclusions, and can be regarded as a branch of applied probability. Most of this chapter 
is about statistics, but Section 11.10 on queueing theory is applied probability. 

When applying statistical methods to a practical problem, the most visible activity is 
the processing of data, using either a hand calculator or, increasingly often, a computer 
statistical package. Either way, a formula or standard procedure from a textbook is 
being applied to the data. The relative ease and obviousness of this activity sometimes 
leads to a false sense that there is nothing more to it. On the contrary, the handling of the 
data (by whatever means) is quite superficial compared with the essential task of trying 
to understand both the problem at hand and the assumptions upon which the various 
statistical procedures are based. If the wrong procedure is chosen, a wrong conclusion 
may be drawn. 

It is, unfortunately, all too easy to use a formula while overlooking its theoretical 
basis, which largely determines its applicability. It is true that there are some statistical 
methods that continue to work reliably even where the assumptions upon which they 
are based do not hold (such methods are called robust), but it is unwise to rely too 
heavily upon this and even worse to be unaware of the assumptions at all. 

The conclusions of a statistical analysis are often expressed in a qualified way such 
as ‘We can be 95% sure that .. .’. At first this seems vague and inadequate. Perhaps a 
decision has to be made, but the statistical conclusion is not expressed simply as ‘yes’ 
or ‘no’. A statistical analysis is rather like a legal case in which the witness is required 
to tell ‘the whole truth and nothing but the truth’. In the present context ‘the whole 
truth’ means that the statistician must glean as much information from the data as is 
possible until nothing but pure randomness remains. ‘Nothing but the truth’ means that 
the statistician must not state the conclusion with any greater degree of certainty or 
confidence than is justified by the analysis. In fact there is a practical compromise 
between truth and precision that will be explained in Section 11.3.3. The result of all 
this is that the decision-maker is aided by the analysis but not pre-empted by it. 

In this chapter we shall first review the basic theory of probability and then cover 
some applications that are beneficial in engineering and many other fields: the statistics 
of means, proportions and correlation, linear regression and goodness-of-fit testing, 
queueing theory and quality control. 


Review of basic probability theory 
This section contains an overview of the basic theory used in the remainder of this 


chapter. No attempt is made to explain or justify the ideas or results: a full account 
can be found in Chapter 13 of Modern Engineering Mathematics or elsewhere. For the 
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same reason there are no examples or exercises. In the process of reviewing the basic 
theory, this section also establishes the pattern of notation used throughout the chapter, 
which follows standard conventions as far as possible. No reader should embark on this 
chapter without having a fairly thorough understanding of the material in this section. 


The rules of probability 


We associate a probability P(A) with an event 4, which in general is a subset of a sample 
space S. The usual set-theoretic operations apply to the events (subsets) in S, and there 
are corresponding rules that must be satisfied by the probabilities. 


Complement rule 
P(S— A) =1-—- P(A) 
The complement of an event 4 is often written as A. 
Addition rule 
P(A U B)= P(A) + P(B) - P(A B) 
For disjoint events, A  B = ©, and the addition rule takes the simple form 
P(A U B)= P(A) + P(B) 
Product rule 
P(A 1 B)=P(A)P(B | A) 


This is actually the definition of the conditional probability P(B | A) of B given A. If A 
and B are independent then the product rule takes the simple form 


P(A M B) = P(A)P(B) 


Random variables 


A random variable has a sample space of possible numerical values together with a 
distribution of probabilities. Random variables can be either discrete or continuous. 
For a discrete random variable (X, say) the possible values can be written as a list 
{V, V>, V3, ... } with corresponding probabilities P(X¥ = v,), P(X = v,), P(X = v3), .... 
The mean of X is then defined as 


Hy= by v;, P(X = vx) 
k 


(sum over all possible values), and is a measure of the central location of the distribution. 
The variance of X is defined as 


Var(X) = oF =} (ve- My) P(X = v4) 


k 


and is a measure of dispersion of the distribution about the mean. The symbols y and 
o° are conventional for these quantities. In general, the expected value of a function 
A(X) of X is defined as 


E{h(X)} = ¥ hv PX = vy) 
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Figure 11.1 
Probability of interval 
from density function. 


Sx @) 


oO xy Xy x 


of which the mean and variance are special cases. The standard deviation Oy is the 
square root of the variance. 
For a continuous random variable (X, say), there is a probability density function 


f(x) and a cumulative distribution function F(x). The cumulative distribution function 


is defined as 
Fy(x) = P(X < x) 


and is the indefinite integral of the density function: 


Fy (x)= | Sx(t) dt 
These functions determine the probabilities of events for the random variable. The 
probability that the variable X takes a value within the real interval (x,, x,) is the area 
under the density function over that interval, or equivalently the difference in values of 
the distribution function at its ends: 
x2 
P(x, <X< x)= | fx(t) dt = Fy (x2) - Fy) 
x} 
(see Figure 11.1). Note that the events x, << ¥ <x, x) <X <x, x, < X¥ <x, and 
x, < X < x, are all equivalent in probability terms for a continuous random variable 
X because the probability of X being exactly equal to either x, or x, is zero. The mean 
and variance of X, and the expected value of a function A(X’), are defined in terms of 
the density function by 


eS | xfix(x) dx 
Vax) =o | (x = fly) fic(x) dx 
E{h(X)} = | h(x) f(x) dx 


These definitions assume that the random variable is defined for values of x from —° to 
co. If the random variable is defined in general for values in some real interval, say 
(a, b), then the domain of integration can be restricted to that interval, or alternatively 
the density function can be defined to be zero outside that interval. 
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Just as events can be independent (and then obey a simple product rule of probab- 
ilities), so can random variables be independent. We shall consider this in more detail in 
Section 11.4. Means and variances of random variables (whether discrete or continuous) 
have the following important properties (Y and Y are random variables, and c is an 
arbitrary constant): 


(CX) = CE X)|="city 
Var(eX)|= ce VarLX) = 62 
E(X+c)=E(X)+c=pyte 
Var(X + c) = Var(X) = of 
E(X + Y) = E(X) + EY) = by + My 

(this applies whether or not X and Y are independent), 
Var(X + Y) = Var(X) + Var(Y) = 07 + oF 


(this applies only when_X and Y are independent). 
It is also useful to note that Var(X ) = E(X’) — [E(X)P. 


The Bernoulli, binomial and Poisson distributions 


The simplest example of a discrete distribution is the Bernoulli distribution. This has 
just two values: X= | with probability p and X = 0 with probability 1 — p, from which 
the mean and variance are p and p(1 — p) respectively. 

The binomial and Poisson distributions are families of discrete distributions whose 
probabilities are generated by formulae, and which arise in many real situations. The 
binomial distribution governs the number (X, say) of ‘successes’ in n independent 
‘trials’, with a probability p of ‘success’ at each trial: 


P(X =k) = (”) pia-py" 


where the range of possible values (A) is {0, 1, 2,...,}. The binomial distribution can 
be thought of as the sum of m independent Bernoulli random variables. This distribution 
(more properly, family of distributions) has two parameters, 7 and p. In terms of these 
parameters, the mean and variance are 


Uy= np 
ox = np(1 — p) 
The Poisson distribution is defined as 


where the range of possible values (f) is the set of non-negative integers {0, 1,2,... }. 
This has mean and variance both equal to the single parameter A (see Section 11.7.1), 
and, by setting A = np, provides a useful approximation to the binomial distribution that 
works when v is large and p is small (see Section 11.7.2). As a guide, the approximation 
can be used when n = 25 and p S 0.1. The Poisson distribution has many other uses, 
as will be seen in Section 11.10. 
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11.2.4 


The normal distribution 


This is a family of continuous distributions with probability density function given by 


fel = BOP Le] 


for —co < x < +c0, where the parameters Wy and oy are the mean and standard deviation 
of the distribution. It is conventional to denote the fact that a random variable X has a 
normal distribution by 


X~ M by, Ox) 


The standard normal distribution is a special case with zero mean and unit variance, 
often denoted by Z: 


Z~ N(O, 1) 


Tables of the standard normal cumulative distribution function 


= Ne awl ° -P/2 
@(z) = P(Z Sz) J(2n) f e dt 


are widely available (see for example Figure 11.2). These tables can be used for 
probability calculations involving arbitrary normal random variables. For example, if 
X ~ N(x, Oy) then 


P(X <a)= p(t fee Bs) 7 o(tks) 
Ox Oy Oy 

The key result for applications of the normal distribution is the central limit theorem: 
if {X, X5, X3,...,X,,} are independent and identically distributed random variables (the 
distribution being arbitrary), each with mean py and variance oy, and if 

W, = as | em eae One 

n Oyyn 

then, as n > ©, the distributions of W, and Z, tend to W,, ~ N(y, Oy /n) and Z, ~ N(O, 1) 
respectively. Loosely speaking, the sum of independent identically distributed random 
variables tends to a normal distribution. 

This theorem is proved in Section 11.7.3, and in the key to many statistical processes, 
some of which are described in Section 11.3. One corollary is that the normal distribution 
can be used to approximate the binomial distribution when z is sufficiently large: if X 
is binomial with parameters n and p then the approximating distribution (by equating 
the means and variances) is Y ~ M(np, np(1 — p)). This is explained (together with the 
important continuity correction) in Section 13.5.5 of Modern Engineering Mathematics, 
and the approximation is used as follows: 


Pxs # ~ o(E+03—27) 


\[np( - p)] 
P(X=h) = o(EL0S—ne) o(£=05 = 22) 
\[npC - p)] \Inp( - p)] 


As a guide, the approximation can be used when n = 25 and 0.1 S p < 0.9. 
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Figure 11.2 Table of 
the standard normal 
cumulative distribution 
function ®(z). 
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Z .00 01 .02 .03 .04 05 .06 .07 .08 .09 


0 5000 5040» .5080S 5120 5160S S199 523952795319 .5359 
1 539854385478 5517 555755965636 5675S 714.5753 
2 5793-5832, S871—s5910— 59485987 = .6026-~— «6064. —Ss«i«103.—s«C 14 
3 6179 6217 ~—.6255. «6293S «6331S «6368S «64066443 «64806517 
4A 6554-6591 .6628 6664. —Ss«w6700~S««6736~— 67726808 ~—.6844. 6879 
5 6915 6950 .6985 .7019 .7054 .7088 7123. 7157S s««7190— 7224 
6 7257 72917324. 7357S 7389 .7422,—s«7454.—Ss«£7486-~— 75177549 
7.7580 7611. —.7642,—Ss«£7673—«.7704.—S«.7734.—.7764.~S 779478237852 
8.7881 .7910 7939-7967 «79958023. s«8051.—Ss«w8078 =~ 8106 S813 
9 8159 8186 8212 8238 = .8264.—S «8289 = 83158340 = 83658389 
1.0 8413 8438 8461 8485 8508 .8531 8554. 8577S 85998621 
1.1 8643. 8665. = «8686. ——8708 ~—S—8729.~S 687498770 ~—S.8790~—S 8810 ~—.8830 
1.2 8849 8869 8888 8907 8925 8944 ~—-.8962—Ss«w8980~—.8997— 9015 
13.9032 9049 ~=—-.9066.-~— «9082S «9099-9115. 9131.— 914791629177 
14.9192 9207. 9222-9236 = 9251-9265. 9279 = .9292 9306 ~——«.9319 
1.5 9332 9345. 9357-9370 s«9382,s«9394.s«9406-—Sw9418 9429944 
1.6 9452 9463 9474 9484 9495 9505 .9515 9525. 9535. .9545 
1.7 9554 9564-9573. 9582s“ 9591 959996089616 96259633 
18 9641 9649 9656 9664 9671 9678  .9686 9693 9699 9706 
1.9 9713. 9719 9726 9732 9738  .9744 .9750 .9756 9761 — .9767 
2.0 .9772 9778 9783 .9788 9793 .9798  .9803. 9808 + = .9812.——.9817 
2.1 9821 9826 9830 9834 9838  .9842 9846 9850 9854 9857 
2.2 9861 9864 9868 9871 9875  .9878  .9881 9884 9887 9890 
2.3 9893 9896 9898 9901 9904 9906 9909 9911 9913 9916 
24 9918 9920 9922 9925 9927 9929 9931 9932 9934 9936 
2.5 9938 9940 9941 9943 9945 9946 9948 9949 9951 9952 
2.6 .9953 9955 9956 .9957 9959 9960  .9961 9962 9963-9964 
2.7 9965 9966 9967 .9968  .9969 9970 .9971 9972 9973 9974 
2.8 9974 9975 9976 .9977 9977 .9978  .9979 9979 9980 .9981 
2.9 9981 9982 9982 9983 9984 9984 9985 9985 9986 9986 
3.0 9987 9987  .9987 9988 9988  .9989 998999899990 .9990 
3.1 9990 9991 9991 9991 9992 9992 9992 9992 9993 9993 
3.2 9993 9993 9994 9994 9994 9994 9994 9995 9995 9995 
33.9995 9995 9995 9996 9996 9996 .9996 .9996 .9996  .9997 
34 9997 9997 9997 9997 9997 9997 9997 9997 9997  .9998 
Zz 1.282 1.645 1.960 2.326 2.576 3.090 3.291 3.891 4.417 
(2) 90 95 975.99 995 999 9995 99995 999995 
2[I- @(2)]_~—«.20 10 05 02 01 002 ~=—-.001 000 1 00001 


Sample measures 


It is conventional to denote a random variable by an upper-case letter (X, say), and an 
actual observed value of it by the corresponding lower-case letter (x, say). An observed 
value x will be one of the set of possible values (sample space) for the random variable, 
which for a discrete random variable may be written as a list of the form {v,, v5, v3, ... }. 
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c Be IP a 


It is possible to observe a random variable many times (say times) and obtain a series 
of values. In this case we assume that the random variable X refers to a population 
(whose characteristics may be unknown), and the series of random variables {X,, X), 
...,X,} as a sample. Each_X; is assumed to have the characteristics of the population, 
so they all have the same distribution. The actual series of values {x,, x,,..., x,} 
consists of data upon which we can work, but it is useful to define certain sample 
measures in terms of the random variables {X,, X,,..., X,} in order to interpret the 
data. Principal among these measures are the sample average and sample variance, 
defined as 


fetyx,  st=t yn - x 
i=l 


respectively, and it is useful to note that the sample variance is the average of the 
squares minus the square of the average: 


Sax uy 


We shall also need the following alternative definition of sample variance in Section 11.3.5: 
1x o\2 
1 a i > Y-X 
X,n-1 n= 1 2 i ) 


We can use the properties of means and variances (summarized in Section 11.2.2) to 
find the mean and variance of the sample average as follows: 


E(®) = 1 EX + ee a TE) + +X] 


= Thy _ 


n 


Var) = 4 Van(X, +... +X) = 4 [vary +... + Var(X,)] 
n n 


Here we are assuming that the population mean and variance are Ly and oy respectively 
(which may be unknown values in practice), and that the observations of the random 
variables X; are independent, a very important requirement in statistics. 


Estimating parameters 


Interval estimates and hypothesis tests 


The first step in statistics is to take some data from an experiment and make inferences 
about the values of certain parameters. Such parameters could be the mean and variance 
of a population, or the correlation between two variables for a population. The data are 
never sufficient to determine the values exactly, but two kinds of inferences can be made: 
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(a) arange of values can be quoted, within which it is believed with high probability 
that the population parameter value lies, or 


(b) a decision can be made as to whether or not the data are compatible with a 
particular value of the parameter. 


The first of these is called interval estimation, and provides an assessment of the value 
that is rather more honest than merely quoting a single number derived from the sample 
data, which may be more or less uncertain depending upon the sample size. The second 
approach is called hypothesis testing and allows a value of particular interest to be 
assessed. These two approaches are usually covered in separate chapters in introductory 
textbooks on statistics, but they are closely related and are often used in conjunction 
with each other. Tests of simple hypotheses about parameter values will therefore be 
covered here within the context of interval estimation. 


Distribution of the sample average 


Suppose that a clearly identified population has a numerical characteristic with an 
unknown mean value, such as the mean lifetime for a kind of electronic component or 
the mean salary for a job category. A natural way to estimate this unknown mean is to 
take a sample from the population, measure the appropriate characteristic, and find the 
average value. If the sample size is n and the measured values are {x,, x,,...,x,} then 
the average value 


n 
= 1 
a1 x, 
n@ 
i=1 


is a reasonable estimate of the population mean Ly provided that the sample is repres- 
entative and independent, and the size n is sufficiently large. 
We can be more precise about how useful this estimate is if we treat the sample 


average as a random variable. Now we have a sample {X,, X5,..., X,} with average 
X=1yy, 
n 


and the mean and variance of X are given by 
2 


E(X)=by, — Var(%) = =i 


(see Section 11.2.5). This shows that the expected value of the average is indeed equal 
to the population mean, and that the variance is smaller for larger samples. However, 
we can go further. The central limit theorem (Section 11.2.4) tells us that sums of ident- 
ical random variables tend to have a normal distribution regardless of the distribution 
of the variables themselves. The only requirement is that a sufficient number of variables 
contribute to the sum (the actual number required depends very much on the shape of 
the underlying distribution). 


The sample average is a sum of random variables, and therefore has (approximately) 
a normal distribution for a sufficiently large sample: 


X~ N( My, Oy/n) 
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Solution 


Figure 11.3 Normal 
density function for 
Example 11.1. 


11.3.3 


This allows us to use a general method of inference concerning means instead of a 
separate method for each underlying distribution — even if this were known, which is 
usually not the case. In practice, a sample size of 25 or more is usually sufficient for the 
normal approximation. 


For all children taking an examination, the mean mark was 60%, with a standard 
deviation of 8%. A particular class of 30 children achieved an average of 63%. Is this 
unusual? 


The average of 63% is higher than the mean, but not by very much. We do not know the true 
distribution of marks, but the sample average has (approximately) a normal distribution. 
We can test the idea that this particular class result is a fluke by reducing the sample 
average to a standard normal in the manner described in Section 11.2.4 and checking 
its value against the table of the cumulative distribution function ®(z) (Figure 11.2): 


X- 60 ~ 63-60 


P(X = 63)=P = = 1 — &(2.054) = 0.020 
¢ ) Ga 8//30 na 


It is unlikely (one chance in 50) that an average as high as this could occur by chance, 
assuming that the ability of the class is typical. Figure 11.3 illustrates that the result 
is towards the tail of the distribution. It therefore seems that this class is unusually 
successful. 


Confidence interval for the mean 


A useful notation will be introduced here. For the standard normal distribution, define 
Z,, to be the point on the z axis for which the area under the density function to its right 
is equal to a: 


P(Z>z,) =a 
or equivalently 


O(z,)=1-a 
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(a) (b) 


Figure 11.4 Normal density functions with (a) z, and (b) Z,;.. 


Example 11.2 


(see Figure 11.4a). From the standard normal table we have Zp 9; = 1.645 and Zp 95 = 1.96. 
By symmetry 


PCZan <Z< Zan) = 1-0 


(see Figure 11.4b). Assuming normality of the sample average, we have 


aN 


P(-Za < a < Zan) =(1- a) 
Oo 
which, after multiplying through the inequality by o,//n and changing the sign, gives 
P(~Zan <py-X< Zan) = l-a@ 
\n \n 
so that 


= oO = 
P(X - zon F< Hy <X+ Zan 8) = l-a 
yn \ 
Assume for now that the standard deviation of X is known (it is actually very rare for 
Oy to be known when ily is unknown, but we shall discuss this case first for simplicity 
and later consider the more general situation where both Ly and oy are unknown). 


The interval defined by (X¥ + z,.0,y/\n) is called a 100(1 — @)% confidence interval 
for the mean, with variance known. Ifa value for @ is specified, the upper and lower 
limits of this interval can be calculated from the sample average. The probability is 
1 — a that the true mean lies between them. 


The temperature (in degrees Celsius) at ten points chosen at random in a large building 
is measured, giving the following list of readings: 


{18°, 16.5°, 17.5°, 18°, 19.5°, 16.5°, 18°, 17°, 19°, 17.5°} 


The standard deviation of temperature through the building is known from past 
experience to be 1°C. Find a 90% confidence interval for the mean temperature in the 
building. 
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Solution 


Figure 11.5 
Confidence intervals: 
(a) infinite interval; 
(b) finite interval; 
(c) point value. 


Example 11.3 


The average of the ten readings is 17.75 °C, and, using Zp 95 = 1.645, the 90% confidence 
interval is 


(17.75 + 1.645(1//10)) = (17.1, 18.3) 


The confidence interval is used to indicate the degree of uncertainty in the sample 
average. The simplicity of the calculation is deceptive because the idea is very important 
and easily misunderstood. It is not the mean that is random but rather the interval that 
would enclose it 100(1 — a)% of the times the experiment is performed. It is tempting 
to think of the interval as fixed by the experiment and the mean as a random variable that 
has a probability 1 — @ of lying within it, but this is not correct. 

Typical values of a are 0.1, 0.05 and 0.01, giving 90%, 95% and 99% confidence 
intervals respectively. The value chosen is a compromise between truth and precision, 
as illustrated in Figure 11.5. A statement saying that the mean lies within the interval 
(—ee, co) is 100% true (certain to be the case), but totally uninformative because of its 
total imprecision. None of the possible values is ruled out. On the other hand, saying 
that the mean equals the exact value given by the sample average is maximally precise, 
but again of limited value because the statement is false — or rather the probability of 
its truth is zero. A statement quoting a finite interval for the mean has a probability of 
being true, chosen to be quite high, and at the same time it rules out most of the possible 
values and therefore is highly informative. The higher the probability of truth, the lower 
the informativeness, and vice versa. 

The width of the interval also depends on the sample size. A larger experiment 
yields a more precise result. If figures for the confidence 1 — @ and precision (width of 
the interval) are specified in advance then the sample size can be chosen sufficiently 
large to satisfy these requirements. In some experimental situations (for example, 
destructive testing) there are incentives to keep sample sizes as small as possible. The 
experimenter must weigh up these conflicting objectives and design the experiment 
accordingly. 


> 
» 
> 


(a) (b) (c) 


A machine fills cartons of liquid; the mean fill is adjustable but the dial on the gauge 
is not very accurate. The standard deviation of the quantity of fill is 6 ml. A sample of 
30 cartons gave a measured average content of 570 ml. Find 90% and 95% confidence 
intervals for the mean. 
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Using a@ = 0.05 and Zp; = 1.96, the 95% confidence interval is 
(570 + 1.960(6//30)) = (567.8, 572.1) 

Likewise, using @ = 0.1 and Z95 = 1.645, the 90% confidence interval is 
(570 + 1.645(6//30)) = (568.2, 571.8) 


As expected, the 95% interval is slightly wider. 


Testing simple hypotheses 


As explained in Section 11.3.1, the testing of hypotheses about parameter values is 

complementary to the estimation process involving an interval. A ‘simple’ hypothesis 

is one that specifies a particular value for the parameter, as opposed to an interval, and 

it is this type that we shall consider. The following remarks apply generally to parameter 

hypothesis testing, but will be directed in particular to hypotheses concerning means. 
There are two kinds of errors that can occur when testing hypotheses: 


(a) a true hypothesis can be rejected (this is usually referred to as a type I error), or 
(b) a false hypothesis can be accepted (this is usually called a type II error). 


In reality, all hypotheses that prescribe particular values for parameters are false, but they 
may be approximately true and rejection may be the result of an experimental fluke. 
This is the sense in which a type I error can occur. Any hypothesis will be rejected if 
the sample size is large enough. Acceptance really means that there is insufficient 
evidence to reject the hypothesis, but this is not an entirely negative view because if the 
hypothesis has survived the test then it has some degree of dependability. 


Normally a simple hypothesis is tested by evaluating a test statistic, a quantity 
that depends upon the sample and leads to rejection of the hypothesized parameter 
value if its magnitude exceeds a certain threshold. If the hypothesized mean is Ly 
then the test statistic for the mean is 


ye Gel 
Ox/l\n 


with the hypothesis ‘rejected at significance level a” if |Z| > Zy;. 


The significance level can be regarded as the probability of false rejection, an error 
of type I. If the hypothesis is true then Z has a standard normal distribution and the 
probability that it will exceed z,,. in magnitude is a. If Z does exceed this value then 
either the hypothesis is wrong or else a rare event has occurred. It is easy to show 
that the test statistic lies on this threshold (for significance level a) exactly when 
the hypothesized value lies at one or other extreme of the 100(1 — a)% confidence 
interval (see Figure 11.6). An alternative way to test the hypothesis is therefore to see 
whether or not the value lies within the confidence interval. 
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Figure 11.6 
Confidence interval 
and hypothesis test. 


Example 11.4 


Solution 


11.3.5 


AS. 


a 
i >| 
Confidence interval 
1 
1 


i 
' 
‘ 
' 


Acceptance interval for jo 


For the situation described in Example 11.3 test the hypothesis that the mean fill of liquid 
is 568 ml (one imperial pint). 


The value of the test statistic is 


z= 510-568 _ 5 94 


6/30 

This exceeds zy); = 1.645 (10% significance), but is less than Zp; = 1.96 (5% signi- 
ficance). Alternatively, the quoted figure lies within the 95% confidence interval but 
outside the 90% confidence interval. Either way, the hypothesis is rejected at the 10% 
significance level but accepted at the 5% level. If the actual mean is 568 ml then there 
is less than one chance in 10 (but more than one in 20) that a result as extreme as 570 ml 
will be obtained. It looks as though the true mean is larger than the intended value, but 
the evidence is not particularly strong. The probability of false rejection (type I error) 
is somewhere between 5% and 10%, which is small but not negligible. 


Examples 11.3 and 11.4 set the pattern for the interpretation and use of confidence 
intervals. We shall now see how to apply these ideas more generally. 


Other confidence intervals and tests concerning means 


Mean when variance is unknown 


With the basic ideas of interval estimation and hypothesis testing established, it is 
relatively easy to cover other cases. The first and most obvious is to remove the assump- 
tion that the variance is known. If the sample size is large then there is essentially no 
problem, because the sample standard deviation S,,_, can be used in place of oy in the 
confidence interval, where 
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Standard normal 


2 1 > 
Sin =—> HK - XY 


i=l 


This definition was introduced in Section 11.2.5. Note that the sum is divided by n — 1 
rather than n. For a large sample this makes little difference, but for a small sample this 
form must be used because the ‘t distribution’ requires it. 

Suppose that the sample size is small, say less than 25. Using Sy,,_, in place of oy 
adds an extra uncertainty because this estimate is itself subject to error. Furthermore, 
the central limit theorem cannot be relied upon to ensure that the sample average has a 
normal distribution. We have to assume that the data themselves are normal. In this 
situation the random variable 


T,= AS Hy 
Sy ni/\n 
has a ¢ distribution with parameter n — |. This distribution resembles the normal dis- 
tribution, as can be seen in Figure 11.7, which shows the density functions of 7, and 
Ts; together with that of the standard normal distribution. In fact 7,, tends to the stand- 
ard normal distribution as n > oo. The parameter of the ¢ distribution (whose value 
here is one less than the size of the sample) is usually called the number of degrees 
of freedom. 


Defining ¢,,,_; by 
JAIL. 2 ie) = OF 
(by analogy with z,), we can derive a 100(1 — a)% confidence interval for the mean 
by the method used in Section 11.3.3: 
[2 tages | 
yn 


This takes explicit account of the uncertainty caused by the use of S,.,_, in place of 
Oy. Values of ¢,,,; for typical values of @ can be read directly from the table of the 
t distribution, an example of which is shown in Figure 11.8. To obtain a test statistic 
for an assumed mean LU, simply replace Uy by My in the definition of T,. 
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Figure 11.8 Table 
of the ¢ distribution ¢ n a=0.10 a= 0.05 a= 0.025 a=0.01 a= 0.005 n 


on" 


(Based on Table 12 


of Biometrika Tables 1 3.078 6.314 12.706 31.821 63.657 1 

for Statisticians, 2 1.886 2.920 4.303 6.965 9.925 2 

Volume 1. Cambridge 3 1.638 2.353 3.182 4.541 5.841 3 

University Press, 1954. 4 1.533 2.132 2.776 3.747 4.604 4 

By permission of the 5 1.476 2.015 2.571 3.365 4.032 5 
Biometrika trustees.) 

6 1.440 1.943 2.447 3.143 3.707 6 

7 1.415 1.895 2.365 2.998 3.499 7 

8 1.397 1.860 2.306 2.896 3.355 8 

9 1.383 1.833 2.262 2.821 3.250 9 

10 1.372 1.812 2.228 2.764 3.169 10 

11 1.363 1.796 2.201 2.718 3.106 11 

12 1.356 1.782 2.179 2.681 3.055 12 

13 1.350 1.771 2.160 2.650 3.012 13 

14 1.345 1.761 2.145 2.624 2.977 14 

15 1.341 1.753 2.131 2.602 2.947 15 

16 1.337 1.746 2.120 2.583 2.921 16 

17 1.333 1.740 2.110 2.567 2.898 17 

18 1.330 1.734 2.101 2.552 2.878 18 

19 1.328 1.729 2.093 2.539 2.861 19 

20 1.325 1.725 2.086 2.528 2.845 20 

21 1.323 1.721 2.080 2.518 2.831 21 

22 1.321 1 AL 2.074 2.508 2.819 22 

23 1.319 1.714 2.069 2.500 2.807 23 

24 1.318 17d 2.064 2.492 2.797 24 

25 1.316 1.708 2.060 2.485 2.787 25 

26 1.315 1.706 2.056 2.479 2.779 26 

27 1.314 1.703 2.052 2.473 2.771 27 

28 1.313 1.701 2.048 2.467 2.763 28 

29 1.311 1.699 2.045 2.462 2.756 29 


oo 1.282 1.645 1.960 2.326 2.576 co 


Example 11.5 —‘ The measured lifetimes of a sample of 20 electronic components gave an average of 
1250h, with a sample standard deviation of 96h. Assuming that the lifetime has a normal 
distribution, find a 95% confidence interval for the mean lifetime of the population, and 
test the hypothesis that the mean is 1300h. 


Solution The appropriate figure from the ¢ table is f) 95 jy = 2.093, so the 95% confidence interval is 
(1250 + 2.093(96)/,20) = (1205, 1295) 


The claim that the mean lifetime is 1300h is therefore rejected at the 5% significance 
level. The same conclusion is reached by evaluating 


7. = 1250 = 1300 _ 
"96/20 


which exceeds fp 975,19 in magnitude. 


-2.33 
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Difference between means 


Now suppose that we have not just a single sample but two samples from different popula- 
tions, and that we wish to compare the separate means. Assume also that the variances 
of the two populations are equal but unknown (the most common situation). Then it can 
be shown that the 100(1 — a@)% confidence interval for the difference U, — L, between 
the means is 


Tse hl 
[x -X, ar tataSea|(t ar 4) 
1 2 


where X, and X, are the respective sample averages, n, and n, are the respective 
sample sizes, Sj and $3 are the respective sample variances (using the ‘n — 1’ form 
as above), 


52 = (= DSi + (m= D)S3 
a Dy te Wy = 2 


is a pooled estimate of the unknown variance, and 
n= ny — 2 
is the parameter for the ¢ table. The corresponding test statistic for an assumed 
difference d) = LM, — My 1s 
af = X, ae xX; an do 
‘ S,V(1/n, + 1/ny) 


For small samples the populations have to be normal, but for larger samples this is not 
required and the t-table figure can be replaced by z,,,.. 


Two kinds of a new plastic material are to be compared for strength. From tensile strength 
measurements of 10 similar pieces of each type, the sample averages and standard 
deviations were as follows: 


X,= 78.3, S,=5.6, X,=84.2, 8,=63 


Compare the mean strengths, assuming normal data. 


The pooled estimate of the standard deviation is 5.960, the ¢ table gives fy 75:3 = 2.101, 
and the 95% confidence interval for the difference between means is 


(78.3 — 84.2 + 2.101(5.96)//5) = (-11.5, —0.3) 


The difference is significant at the 5% level because zero does not lie within the interval. 
Also, assuming zero difference gives 


r, = 183-842 __y 9) 


5.96/,/5 
which confirms the 5% significance. 
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It is also possible to set up confidence intervals and tests for the variance oy, or for 
comparing two variances for different populations. The process of testing means and 
variances within and between several populations is called analysis of variance. This 
has many applications, and is well covered in statistics textbooks. 


Interval and test for proportion 


The ideas of interval estimation do not just apply to means. If probability is interpreted 
as a long-term proportion (which is one of the common interpretations) then measuring 
a sample proportion is a way of estimating a probability. The binomial distribution 
(Section 11.2.3) points the way. We count the number of ‘successes’, say_X, inn ‘trials’, 
and estimate the probability p of success at each trial, or the long-term proportion, by 
the sample proportion 


(it is common in statistics to place the ‘hat’ symbol * over a parameter to denote an 
estimate of that parameter). This only provides a point estimate. To obtain a confidence 
interval, we can exploit the normal approximation to the binomial (Section 11.2.4) 


X ~ N(np, np(\ - p)) 


approximately, for large n. Dividing by n preserves normality, so 
4 1- 
ae N(»,@ - 2) 


Following the argument in Section 11.3.3, we have 


Poza | [PASE] <p < peers [|e =l-a 
7 n 


and, after rearranging the inequality, 


P p-tens| [22] <p < P+teny| [Oe =l-@ 
he n 


Because p is unknown, we have to make a further approximation by replacing p by p 
inside the square root, to give an approximate 100(1 — @)% confidence interval for p: 


jn f= 


The corresponding test statistic for an assumed proportion py is 


os X= NP 


~ \[npo(1 - po) 


with +z, as the rejection points for significance level a. 
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In an opinion poll conducted with a sample of 1000 people chosen at random, 30% said 
that they support a certain political party. Find a 95% confidence interval for the actual 
proportion of the population who support this party. 


The required confidence interval is obtained directly as 


0.3 + 196 /| Te = (0.27, 0.33) 


A variation of about 3% either way is therefore to be expected when conducting opinion 
polls with sample sizes of this order, which is fairly typical, and this figure is often 
quoted in the news media as an indication of maximum likely error. 


A similar argument that also exploits the fact that the difference between two 
independent normal random variables is also normal leads to the following 100(1 — a)% 
confidence interval for the difference between two proportions, when p, and p, are 
the respective sample proportions: 


Ss Onl ye 5(1-p 
en Pi) , Bal P2) 


ny Ny 
Again it is assumed that n, and n, are reasonably large. The test statistic for equality 
of proportions is 
Z= Pi = P2 
V[pC - p)(1/n, + 1/ny)] 
where p = (X, +-X,)/(n, + n>) is a pooled estimate of the proportion. 


One hundred samples of an alloy are tested for resistance to fatigue. Half have been 
prepared using a new process and the other half by a standard process. Of those pre- 
pared by the new process, 35 exhibit good fatigue resistance, whereas only 25 of those 
prepared in the standard way show the same performance. Is the new process better 
than the standard one? 


The proportions of good samples are 0.7 for the new process and 0.5 for the standard 
one, so a 95% confidence interval for the difference between the true proportions is 


0.7-0.5+ 1964] 2S ¥ ns = (0.01, 0.39) 


The pooled estimate of proportion is 


p= (35 + 25)/(50 + 50) = 0.6 
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so that 


ee ae 
\[(0.6)(0.4)/25] 


Both approaches show that the difference is significant at the 5% level. However, it is 
only just so: if one more sample for the new process had been less fatigue-resistant, the 
difference would not have been significant at this level. This suggests that the new process 
is effective — but, despite the apparently large difference in success rates, the evidence 
is not very strong. 


This method only applies to independent sample proportions. It would not be legitim- 
ate to apply it, for instance, to a more elaborate version of the opinion poll (Example 11.7) 
in which respondents can choose between two (or more) political parties or else support 
neither. Support for one party usually precludes support for another, so the proportions 
of those interviewed who support the two parties are not independent. More elaborate 
confidence intervals, based on the multinomial distribution, can handle such situations. 
This shows how important it is to understand the assumptions upon which statistical 
methods are based. It would be very easy to look up ‘difference between proportions’ 
in an index and apply an inappropriate formula. 


11.3.7 Exercises 


An electrical firm manufactures light bulbs whose 
lifetime is approximately normally distributed with 
a standard deviation of 50h. 


(a) Ifa sample of 30 bulbs has an average life of 
780 h, find a 95% confidence interval for the 
mean lifetime of the population. 

(b) How large a sample is needed if we wish to be 
95% confident that our sample average will be 
within 10h of the population mean? 


Monthly rainfall measurements (in mm) were taken 
at a certain location for three years, with results 
as follows: 


38 48 50 94 105 53 81 91 110 103 90 84 
115 113 35 130 77 67 72 113 98 37 61 91 
9 112 29 16 56 61 82 132 48 68 114 55 


Find the average monthly rainfall for this period. 
Also find a 95% confidence interval for the mean 
monthly rainfall, using the measured standard 
deviation as an estimate of the true value. 


Quantities of a trace impurity in 12 specimens of a 
new material are measured (in parts per million) 
as follows: 


8.8, 7.1, 7.9, 10.2, 8.9, 7.7, 10.6, 9.4, 9.2, 7.5, 
9.0, 8.4 


Find a 95% confidence interval for the population 
mean, assuming that the distribution is normal. 


A sample of 30 pieces of a semiconductor material 
gave an average resistivity of 73.2 mQm, with a 
sample standard deviation of 5.4mQ m. Obtain 

a 95% confidence interval for the resistivity of 
the material, and test the hypothesis that this is 

75 mQ m. 


The mean weight loss of 16 grinding balls after a 
certain length of time in mill slurry is 3.42 g, with 
a standard deviation of 0.68 g. Construct a 99% 
confidence interval for the true mean weight 

loss of such grinding balls under the stated 
conditions. 


While performing a certain task under simulated 
weightlessness, the pulse rate of 32 astronaut 
trainees increased on the average by 26.4 beats per 
minute, with a standard deviation of 4.28 beats per 
minute. Construct a 95% confidence interval for the 
true average increase in the pulse rate of astronaut 
trainees performing the given task. 
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The quality of a liquid being used in an etching 
process is monitored automatically by measuring 
the attenuation of a certain wavelength of light 
passing through it. The criterion is that when the 
attenuation reaches 58%, the liquid is declared as 
‘spent’. Ten samples of the liquid are used until 
they are judged as ‘spent’ by the experts. The light 
attenuation is then measured, and gives an average 
result of 56%, with a standard deviation of 3%. 

Is the criterion satisfactory? 


A fleet car company has to decide between 

two brands A and B of tyre for its cars. An 
experiment is conducted using 12 of each brand, 
run until they wear out. The sample averages 
and standard deviations of running distance 

(in km) are respectively 36 300 and 5000 for A, 
and 39 100 and 6100 for B. Obtain a 95% 
confidence interval for the difference in means, 
assuming the distributions to be normal, and test 
the hypothesis that brand B tyres outrun brand 
A tyres. 


A manufacturer claims that the lifetime of a 
particular electronic component is unaffected by 
temperature variations within the range 0—60 °C. 
Two samples of these components were tested, 
and their measured lifetimes (in hours) recorded 
as follows: 


0°C: 7250, 6970, 7370, 7910, 6790, 6850, 
7280, 7830 


60°C: 7030, 7270, 6510, 6700, 7350, 6770, 
6220, 7230 


Assuming that the lifetimes have a normal 
distribution, find 90% and 95% confidence 
intervals for the difference between the mean 
lifetimes at the two temperatures, and hence test 
the manufacturer’s claim at the 5% and 10% 
significance levels. 
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Suppose that out of 540 drivers tested at random, 38 
were found to have consumed more than the legal 
limit of alcohol. Find 90% and 95% confidence 
intervals for the true proportion of drivers who were 
over the limit during the time of the tests. Are the 
results compatible with the hypothesis that this 
proportion is less than 5%? 


It is known that approximately one-quarter of 

all houses in a certain area have inadequate loft 
insulation. How many houses should be inspected 
if the difference between the estimated and true 
proportions having inadequate loft insulation is 
not to exceed 0.05, with probability 90%? If in fact 
200 houses are inspected, and 55 of them have 
inadequate loft insulation, find a 90% confidence 
interval for the true proportion. 


A drug-manufacturer claims that the proportion 
of patients exhibiting side-effects to their new 
anti-arthritis drug is at least 8% lower than 

for the standard brand X. In a controlled 
experiment 31 out of 100 patients receiving 

the new drug exhibited side-effects, as did 74 
out of 150 patients receiving brand X. Test the 
manufacturer’s claim using 90% and 95% 
confidence intervals. 


Suppose that 10 years ago 500 people were 
working in a factory, and 180 of them were 
exposed to a material which is now suspected 
as being carcinogenic. Of those 180, 30 have 
since developed cancer, whereas 32 of the other 
workers (who were not exposed) have also since 
developed cancer. Obtain a 95% confidence 
interval for the difference between the 
proportions with cancer among those exposed 
and not exposed, and assess whether the material 
should be considered carcinogenic, on this 
evidence. 


Joint distributions and correlation 


Just as it is possible for events to be dependent upon one another in that information to 
the effect that one has occurred changes the probability of the other, so it is possible for 
random variables to be associated in value. In this section we show how the degree of 
dependence between two random variables can be defined and measured. 
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11.4.1 Joint and marginal distributions 


The idea that two variables, each of which is random, can be associated in some way 
might seem mysterious at first, but can be clarified with some familiar examples. For 
instance, if one chooses a person at random and measures his or her height and weight, 
each measurement is a random variable — but we know that taller people also tend to 
be heavier than shorter people, so the outcomes will be related. On the other hand, a 
person’s birthday and telephone number are not likely to be related in any way. In 
general, we need a measure of the simultaneous distribution of two random variables. 


For two discrete random variables X and Y with possible values {u,,..., u,,} and 
{v,,..., V,} respectively, the joint distribution of X and Y is the set of all joint 
probabilities of the form 

ROX 0 1) Wem (b= lle cae 5 MB/S My 505 oA) 


The joint distribution contains all relevant information about the random variables 
separately, as well as their joint behaviour. To obtain the distribution of one variable, 
we sum over the possible values of the other: 


PUX= 0) = PX uOY=v) (k=1,...,m) 
PUY =u)= ¥ P(X=u,N Y=0) (j=1,...,n) 


k=1 


The distributions obtained in this way are called marginal distributions of X and Y. 


Example 11.9 Two textbooks are selected at random from a shelf containing three statistics texts, two 
mathematics texts and three engineering texts. Denoting the number of books selected 
in each subject by S, M and E respectively, find (a) the joint distribution of S and M, 
and (b) the marginal distributions of S, M and E. 


Solution (a) 


Figure 11.9 
Joint distribution M 
for Example 11.9. 
S 0 1 2 Total 
0 os 7 7 a 
9 15 
1 8 id 28 
2 3 3 
Total 8 3 + 1 


Bl 
= 
5 
8 


The joint distribution (shown in Figure 11.9) is built up element by element using 
the addition and product rules of probability as follows: 


P(S=M=0)=P(E=2)=(3)3)=3 


28 
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that is, the probability that the first book is an engineering text (three chances out 
of eight) times the probability that the second book is also (two remaining chances 
out of seven). Continuing, 


P(S=0N M=1)=PM=1N E=1) 
=(0)+ @G@)=2 
that is, the probability that the first book is a mathematics text and the second an 


engineering text, plus the (equal) probability of the books being the other way 
round. The other probabilities are derived similarly. 


(b) The marginal distributions of S and M are just the row and column totals as shown 
in Figure 11.9. The marginal distribution of £ can also be derived from the table: 


P(E =2)=P(S=M=0)=% 
PE=1)=PS=1NM=0)+PSs0N0M=1l=2 

P(E =0)=P(S=2)+ PS=1N M=1)+ P=2)=3 

This is the same as the marginal distribution of S, which is not surprising, because 
there are the same numbers of engineering and statistics books on the shelf. 


In order to apply these ideas of joint and marginal distributions to continuous random 
variables, we need to build on the interpretation of the probability density function. 
The joint density function of two continuous random variables X and Y, denoted by 
Sy(x, y), is such that 


25) fae) 
P@,<X<x, and y <Y¥<y,)= | | Fx, v(x, y) dy dx 
TP 
for all intervals (x,, x.) and (y;, y2). This involves a double integral over the two variables 
x and y. This is necessary because the joint density function must indicate the relative 
likelihood of every combination of values of X and Y, just as the joint distribution 
does for discrete random variables. The joint density function is transformed into a 
probability by integrating over an interval for both variables. The double integral here 
can be regarded as a pair of single-variable integrations, with the outer variable (x) held 
constant during the integration with respect to the inner variable (y). In fact the same 
answer is obtained if the integration is performed the other way around. 


The marginal density functions for X and Y are obtained from the joint density 
function in a manner analogous to the discrete case: by integrating over all values 
of the unwanted variable: 


Ix) = | Ixy(%y)dy (-%9 <x < ©) 


Sry) = | Sy(% y)dx  (-09 <y <0) 
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Example 11.10 = The joint density function of random variables X and Y is 


1 OSxS<l,cexSyScx+l1) 


0 otherwise 


Tey, y= 


where c is a constant such that 0 < c < 1 (which means that f, (x, y) is unity over the 
trapezoidal area shown in Figure 11.10 and zero elsewhere). Find the marginal distribu- 
tions of X and Y. Also find the probability that neither X nor Y exceeds one-half, 
assuming c = 1. 


Solution To find the marginal distribution of X, we integrate with respect to y: 


cx+1 
dv=1 (0<SxS1l 
fel) = iN ak ee 
0 otherwise 


The marginal distribution for Y is rather more complicated. Integrating with respect to 
x and assuming that0 <c <1, 


iol gyal) (1272 ie0 
Cc 


ysl) 
+ (0<y<c) 
c 


Figure 11.10 
Density function for Sv(y) =41 (cS 
Example 11.10. 


(Exercise 16). When c = 0, the marginal distribution for Y is the same as that for X. 
Finally, when c = 1, 


1/2 fl/2 1/2 
P(X =<} and rsp=| | Laver =| (L-x)dx=; 


0 x 0 


Here the inner integral (with respect to y) is performed with x treated as constant, and 
the resulting function of x is integrated to give the answer. 


The definitions of joint and marginal distributions can be extended to any number of 
random variables. 


11.4.2 Independence 


The idea of independence of events can be extended to random variables to give us the 
important case in which no information is shared between them. This is important in 
experiments where essentially the same quantity is measured repeatedly, either within a 
single experiment involving repetition or between different experiments. As mentioned 
before, independence within a sample is one of the properties that qualifies the sample 
for analysis and conclusion. 
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Example 11.11 


Solution 


11.4.3 


Two random variables X and Y are called independent if their joint distribution 
factorizes into the product of their marginal distributions: 


PX=u,N Y=v)=P(X=u,)P(Y=v,) in the discrete case 


trv ™ VY =fo AW) in the continuous case 


For example, the random variables XY and Y in Example 11.10 are independent if and 
only ifc=0. 


The assembly of a complex piece of equipment can be divided into two stages. The 
times (in hours) required for the two stages are random variables (X and Y, say) with 
density functions e~ and 2 e~*’ respectively. Assuming that the stage assembly times are 


independent, find the probability that the assembly will be completed within four hours. 


The assumption of independence implies that 


Fey, ¥) =f) f(y) = 2 ev) 
If the time for the first stage is x, the total time will not exceed four hours if 
Y<4-x 


so the required value is 


4 (4-x 4 (4-x 
P(X+Y<4)= | | fix yidyde = | | 26°) dy dx 


0/0 0/0 


4 
= | (e*-e °”) dx = 0.964 


0 


Where random variables are dependent upon one another, it is possible to express 
this dependence by defining a conditional distribution analogous to conditional 
probability, in terms of the joint distribution (or density function) and the marginal dis- 
tributions. Instead of pursuing this idea here, we shall consider a numerical measure of 
dependence that can be estimated from sample data. 


Covariance and correlation 


The use of mean and variance for a random variable is motivated partly by the difficulty 
in determining the full probability distribution in many practical cases. The joint dis- 
tribution of two variables presents even greater difficulties. Since we already have 
numerical measures of location and dispersion for the variables individually, it seems 
reasonable to define a measure of association of the two variables that is independent 
of their separate means and variances so that the new measure provides essentially new 
information about the variables. 

There are four objectives that it seems reasonable for such a measure to satisfy. Its 
value should 
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(a) be zero for independent variables, 
(b) be non-zero for dependent variables, 


(c) indicate the degree of dependence in some well-defined sense, detached from the 
individual means and variances, 


(d) be easy to estimate from sample data. 


It is actually rather difficult to satisfy all of these, but the most popular measure of 
association gets most of the way. 


The covariance of random variables X and Y, denoted by Cov(X, Y ), is defined as 


Cov(X, Y) = E{(X— uy)(Y—- by)} 


y x (uy - Ly)(v; - Wy) P(X = uy, MN Y=v;) 


k=1 j=l 


| | (x - My) (y - Hy) fe y (& y) dx dy 


for discrete and continuous variables respectively. The correlation p, , is the covariance 
divided by the product of the standard deviations: 


Waa 
es OyOy 


If whenever the random variable X is larger than its mean the random variable Y also tends 
to be larger than its mean then the product (X¥ — y)(Y — Uy) will tend to be positive. 
The same will be true if both variables tend to be smaller than their means simultan- 
eously. The covariance is then positive. A negative covariance implies that the variables 
tend to move in opposite directions with respect to their means. Both covariance and 
correlation therefore measure association relative to the mean values of the variables. 
It turns out that correlation measures association relative to the standard deviations 
as well. 

It should be noted that the variance of a random variable X is the same as the 
covariance with itself: 


Var(X ) = Cov(X, X) 


Also, by expanding the product within the integral or sum in the definition of covariance, 
it is easy to show that an alternative expression is 


Cov(X, Y) = E(XY) - E(X)E(Y) 


Although the sign of the covariance indicates the direction of the dependence, its 
magnitude depends not only on the degree of dependence but also upon the variances 
of the random variables, so it fails to satisfy the objective (c). In contrast, the correlation 
is limited in range 


=1 pry = +1 
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Solution 
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and it adopts the limiting values of this range only when the random variables are 
linearly related: 


Pxy=+l_ if and only if there exist a, b such that Y=aX +b 


(this is proved in most textbooks on probability theory). The magnitude of the correlation 
indicates the degree of linear relationship, so that objective (c) is satisfied. 


Find the correlation of the random variables S and M in Example 11.9. 


The joint and marginal distributions of S and M are shown in Figure 11.9. First we find 
the expected values of S and S° from the marginal distribution, and hence the variance 
and standard deviation: 
2 
E(S)=8+(2)(2)=4, (SS) =8+ (42) =2 
Var(S) = 2- (4) 
from which 
Os= 3 35 
Next we do the same for M: 
E(M) = 5+ (2)(53) = 3; E(M*) =3+ (4)(4) =$ 
Var(M) =4-}=3 
from which 
Ou= a ; 


All products of S and M are zero except when both are equal to one, so the expected 
value of the product is 


E(SM) = 33 
The correlation now follows easily: 
dey = ESM = EDEM) _ ig~ GIG) _ 
mm sy w39)G) V9 


The correlation is negative because if there are more statistics books in the selection 
then there will tend to be fewer mathematics books, and vice versa. 


Find the correlation of the random variables XY and Y in Example 11.10. 


Proceeding as in Example 11.12, we have for X 


x= | xdx= 5 


0 


1 
E(X?) = | x dea 


0 
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so that Var(X) = E(X’) — [E(X)> = +. Also, for Y 


12 


‘ic 2 1 lte 
z=| Save] var+| »fi-Fo-d & 
c 1 


0 


= ba 1+c) after simplification 


" 3 1 Ite 
e)=| Zay+| rays| v[t-2o-p]e 
1 


0 é 


4(14¢’?)+4c after simplification 
3 2 p 


so that Var(Y) = E(Y*) — [E(Y)]? = 4(1 +c’). For the expected value of the product 
we have 


1 cx+1 1 
san=| | svdyar=3 x(1+2ex)dx=i+ic 


0 J cx 0 


Finally, the correlation between X and Y is 


_ EXY)- EX)E(Y) 


Px" War (X) Var (Y)] 
_atse7 ate) _ és 
pite) (i te’) 


Note that in fact the result of Example 11.13 holds for any value of c, and not just for 
the range 0 < c < | assumed in Example 11.10. As the value of c increases (positive 
or negative), the correlation increases also, but its magnitude never exceeds one. It is 
also clear that if X and Y are independent then c = 0 and the correlation is zero. Refer 
to Figure 11.10 for a geometrical interpretation. When c = 0, the sample space is a 
square within which all points are equally likely, so there is no association between 
the variables. As c increases (positive or negative), the sample space becomes more 
elongated as the variables become more tightly coupled to one another. 


The general relationship between independence and correlation is expressed as 
follows: if the random variables _X and Y are independent then their correlation is zero. 
This is easily shown as follows for continuous random variables (or by a similar argu- 
ment for discrete random variables). First we have 


Tv, ¥) = fe) fry) 


and then 


| | (x — My)(y - My) fe) f(y) dx dy 


= | (x - Ly) f(x) a (y- Hy) fry) dy = (Ly — My)(Uy — Hy) = 0 
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Unfortunately, the converse does not hold: zero correlation does not imply independence. 
In general, correlation is a measure of linear dependence, and may be zero or very small 
for variables that are dependent in a nonlinear way (see Exercise 15). Objective (a) is 
satisfied, therefore, but not objective (b) in general. 

Another problem with correlation is that a non-zero value does not imply the pres- 
ence of a causal relationship between the variables or the phenomena that they measure. 
Correlation can be ‘spurious’, deriving from some third variable that may be unrecog- 
nized at the time. For example, among the economic statistics that are gathered together 
from many countries and published, there are figures for the number of telephones per 
head of population, birth rate, and the gross domestic product per capita (GDP). It turns 
out that there is a large negative correlation between number of telephones per head and 
the birth rate, but no-one would suggest that telephones have any direct application in 
birth control. The GDP is a measure of wealth, and there is a large positive correlation 
between this and the number of telephones per head, and a large negative correlation 
between GDP and birth rate, both for quite genuine reasons. The correlation between 
telephones and birth rate is therefore spurious, and a more sophisticated measure called 
the partial correlation can be used to eliminate the third variable (provided that it is 
recognized and measured). 

We have considered all the objectives except (d); that this is satisfied is shown in 
Section 11.4.4. 


Sample correlation 


There are two kinds of situations where we take samples of values of two random 
variables X and Y. First we might be interested in the same property for two different 
populations. Perhaps there is evidence that the mean values are different, so we take 
samples of each and compare them. This situation was discussed in Section 11.3.5. The 
second kind involves two different properties for the same population. It is to this situ- 
ation that correlation applies. We take a single sample from the population and measure 


the pair of random variables (X,, Y,) for eachi=1,..., 7. 
For a sample {(X;, Y,),..., (X,, Y,,)} the sample correlation coefficient is 
defined as 


YG -D- PI 


Sy Sy 


Like the true correlation, the sample correlation is limited in value to the range [—1, 1] 
and ry y= +1 when (and only when) all of the points lie along a line. Figure 11.11 contains 
four typical scatter diagrams of samples plotted on the (x, y) plane, with an indication 
of the correlation for each one. The range of behaviour is shown from independence 
(a) through imperfect correlation (b) and (c) to a perfect linear relationship (d). 

By expanding the product within the outer bracket in the numerator, it is easy to 
show that an alternative expression is 
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(c) (d) 


Figure 11.11 Scatter plots for two random variables: (a) Pyy = 0; (b) Pxy > 9; (c) Pxy < 0; (d) Pyy= 1. 


Figure 11.12 it  Toyaereng t 1 lati 
Pseudocadé listing for t Togram to compute sample correlation. 


sample correlation. x(k) and y(k) are the arrays of data, 
n is the sample size, 
xbar and ybar are the sample averages, 
sx and sy are the sample standard deviations, 
rxy is the sample correlation, 
Mx, My, Qx, Qy, Qxy hold running totals. } 


Mx < 0; My — 0 
Qx — 0; Qy — 0 
Qxy — 0 
for k is 1 ton do 
diffx < x(k) — Mx 
diffy <— y(k) — My 
Mx < ((k — 1)*Mx + x(k))/k 
My < ((k — 1)*My + y(k))/k 
Qx — Qx + (1 — I/k)*diffx*diffx 
Qy — Qy + (1 — Ik) *diffy*diffy 
Qxy <— Qxy + (1 — 1/k)*diffx*diffy 
endfor 
xbar — Mx; ybar + My 
sx <— sqr(Qx/n); sy <— sqr(Qy/n) 
rxy <— Qxy/(n*sx*sy) 


which is useful for hand calculation. For computer calculation the best method involves 
the successive sums of products: 


Ov x = Y (xX; = My. )Y; ae My.) 


i=1 


where 


and then ryy = QOyy,,/nSySy. The pseudocode listing in Figure 11.12 exploits the recur- 
rence relation 


Oera= Orns + (1-3) le Meade Mead 


which allows for a single pass through the data with no loss of accuracy. 
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11.4.5 


Example 11.15 
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A material used in the construction industry contains an impurity suspected of having 
an adverse effect upon the material’s performance in resisting long-term operational 
stresses. Percentages of impurity and performance indexes for 22 specimens of this 
material are as follows: 


% Impurity X; 44 55 42 30 45 49 46 $50 47 5.1 4.4 
Performance Y, | 12 14 18 35 23 29 16 12 18 21 27 


%ImpurityX, | 41 49 47 50 46 36 49 SI 48 52 52 
Performance Y¥, | 13. 19 22 20 16 27 «221 4 4613)0©6«©«18)~6«17~(O1 


Find the sample correlation coefficient. 


The following quantities are easily obtained from the data: 
X=4.6545, S,=0.55081, Y=19.1818, S,=6.0350, XY = 87.3591 


(Note that it is advisable to record these results to several significant digits in order to avoid 
losing precision when calculating the difference within the numerator of ryy.) The 
sample correlation is then ry y= —0.58, the negative value suggesting that the impurity 
has an adverse effect upon performance. It remains to be seen whether this is statistic- 
ally significant. 


Interval and test for correlation 


Correlation is more difficult to deal with than mean and proportion, but for normal 
random variables X and Y with a true correlation p, » the sample statistic 


Te ins Ie CU +ryy)C = pry) 
(1 -ryy)(1 + pyy) 


is approximately standard normal for large n. This can be used directly as a test 
statistic for an assumed value of py, y. Alternatively, an approximate 100(1 — a)% 
confidence interval for py y can be derived: 


Geasroscess pee On) 


lt+rtce(1-r) 1+rt+(1-r/e 


where 


2Z 4/2 
C= ex 
aK - ‘| 


(the subscripts XY and Y have been dropped from ry y in this formula). 


For the data in Example 11.14 find 95% and 99% confidence intervals for the true cor- 
relation between percentage of impurity and performance index, and test the hypothesis 
that these are independent. 
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Solution 


11.4.6 


The sample correlation (from the 22 specimens) was found in Example 11.14 to be 
—0.58. For the 95% confidence interval the constant c = 2.458 and the interval itself is 
(—0.80, —0.21). Similarly, the 99% confidence interval is (—0.85, —0.07). Assuming 
Pyy = 0, the value of the test statistic is Z = —2.89, which exceeds Z 9; = 2.576 in 
magnitude. Either way, we can be more than 99% confident that the impurity has an 
adverse effect upon performance. 


Rank correlation 


As has been previously emphasized the correlation only works as a measure of depend- 
ence if 

(1) nis reasonably large, 

(2) Xand Y are numerical characteristics, 

(3) the dependence is Jinear, and 

(4) Xand Y each have a normal distribution. 

There is an alternative form of sample correlation, which has greater applicability, 
requiring only that 

(1) nis reasonably large, 

(2) Xand Y are rankable characteristics, and 

(3) the dependence is monotonic (that is, always in the same direction, which may be 


forward or inverse, but not necessarily linear). 


The variables X and Y can have any distribution. For a set of data X,,..., X,,, a rank of 1 
is assigned to the smallest value, 2 to the next-smallest and so on up to a rank of n assigned 
to the largest. This applies wherever the values are distinct. Tied values are given the 
mean of the ranks they would receive if slightly different. The following is an example: 


oo 
Ww 
Nn 
ao 
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\o 
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Nn 
Ww 
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— 
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The Spearman rank correlation coefficient 7, for data (X;, Y,),..., (X,, Y,,) 1s the 
correlation of the ranks of X; and Y,, where the data X\,..., X, and Y;,..., Y,, are 
ranked separately. If the number of tied values is small compared with n then 


n 


6 2 
ee a 
: mest 


i=l 


where d; is the difference between the rank of X, and that of Y;, The value of r, 
always lies in the interval [—1, 1], and adopts its extreme values only when the 
rankings precisely match (forwards or in reverse). 
To test for dependence, special tables must be used for small samples (n < 20), 
but for larger samples the test statistic 
Z=rg\(n - 1) 


is approximately standard normal. 
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Solution 
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Find and test the rank correlation for the data in Example 11.14. 


The data with their ranks are as follows: 


X; 4.4 5.5 4.2 3.0 4.5 4.9 4.6 5.0 4.7 5.1 4.4 
Rank 5.5 22 4 1 7 14 85 165 10.5 18.5 5.5 


Y, 12 14 18 35 23 29 16 12 18 21 Dt 
Rank 25 6 11 22 18 21 7.5 2.5 11 15.5 19.5 


X; 4.1 4.9 4.7 5.0 4.6 3.6 4.9 5.1 4.8 5.2 5.2 
Rank 3 14 10.5 16.5 8.5 2 14 18.5 12 20.5 20.5 


Y, 13 19 22 20 16 27 21 13 18 17 11 
Rank 45 13 17 14 TO W9S° ~ 155 45 11 9 1 


From this, the rank correlation is rs = —0.361, and Z = —1.66, which exceeds Zo; = 
1.645 and is therefore just significant at the 10% level. If the approximate formula is 
used, the sum of squares of differences is 2398, so 


a p= (OES) 9.954 
(22)(483) 


and Z = —1.62, which is just short of significance. 

These results show that the rank correlation is a more conservative test than the 
sample correlation ry y, in that a larger sample tends to be needed before the hypothesis 
of independence is rejected. A price has to be paid for the wider applicability of the 
method. 


iS 


11.4.7. Exercises 


Suppose that the random variables X and Y have the 15 ~~ Consider the random variable X with density 
following joint distribution: function 
1 (-i<x<t} 
fx) = (-3 * 3) 
X 0 otherwise 
y 1 2 3 Show that the covariance of X and X” is zero. 
(This shows that zero covariance does not imply 
1 0 0.17 0.08 independence, because obviously X* is dependent 
2 0.20 0.11 0 on X.) 
3 0.14 0.25 0.05 
16 ~The joint density function of random variables 


Find (a) the marginal distributions of X and Y, 
(b) P(Y = 3 |X = 2), and (c) the mean, variance 
and correlation coefficient of X and Y. 
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0 otherwise 
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where c is a constant such thatO Sc S 1. 
Find the marginal density function for Y 
(see Example 11.10). 


Let the random variables X and Y represent the 
lifetimes (in hundreds of hours) of two types 
of components used in an electronic system. 
The joint density function is given by 


1 —(x+y)/2 


= x > > 
felt») = ti a i ae. 
0 otherwise 


Find (a) the probability that two components (one 
of each type) will each last longer than 100h, and 
(b) the probability that a component of the second 
type (Y) will have a lifetime in excess of 200 h. 


The following are the measured heights and 
weights of eight people: 


182.8 162.5 175.2 185.4 170.1 167.6 177.8 172.7 
86.1 583 83.0 924 60.2 69.3 83.6 72.7 


Height (cm) 
Weight (kg) 


Find the sample correlation coefficient. 


The number of minutes it took 10 mechanics to 
assemble a piece of machinery in the morning (X ) 
and in the late afternoon (Y) were measured, with 
the following results: 


X/11.1 10.3 
Y|}10.9 14.2 


12.0 15.1 13.7 185 17.3 14.2 14.8 15.3 
13.8 21.5 13.2 21.1 164 19.3 17.4 19.0 


Find the sample correlation coefficient. 


If the sample correlation between resistance and 
failure time for 30 overloaded resistors is 0.7, find a 
95% confidence interval for the true correlation. 


Regression 


Zl 


Ze 


23 


24 


Find a 95% confidence interval for correlation 
between height and weight using the data in 
Exercise 18. 


Marks obtained by 20 students taking examinations 
in mathematics and computer studies were as 
follows: 


Math. | 45 77 43 64 58 64 58 54 71 45 
57 52 67 57 54 54 61 58 55 42 
Comp. | 64 67 47 75 42 65 58 42 70 44 
44 67 49 70 51 58 37 60 42 36 


Find the sample correlation coefficient and the 
90% and 95% confidence intervals. Hence test the 
hypothesis that the two marks are independent at 
the 5% and 10% significance levels. Also find and 
test the rank correlation. 


Let the random variables X and Y have joint density 
function given by 


c(1-y) 
0 otherwise 


(OSxSyS1) 


Sy, y) = 


Find (a) the value of the constant c, (b) P(x < 3 5 
y> 5), and (c) the marginal density functions for 
Xand Y. 


The ball and socket of a joint are separately moulded 
and then assembled together. The diameter of the 
ball is a random variable XY between 29.8 and 

30.3 mm, all values being equally likely. The 
internal diameter of the socket is a random variable 
Y between 30.1 and 30.6 mm, again with all values 
equally likely. The condition for an acceptable 
fit is thatO < Y—X < 0.6 mm. Find the probability 
of this condition being satisfied, assuming that the 
random variables are independent. 


A procedure that is very familiar to engineers is that of drawing a good straight line 
through a set of points on a graph. When calibrating a measuring instrument, for example, 
known inputs are applied, the readings are noted and plotted, a straight line is drawn as 
close to the points as possible (there are bound to be small errors, so they will not all 
lie on the line), and the graph is then used to interpret the readings for unknown inputs. 
It is possible to draw the line by eye, but there is a better way, which involves calculating 
the slope and intercept of the line from the data. The given line then minimizes the total 
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Figure 11.13 Scatter 
plot with regression 
line (Example 11.17). 
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squared error for the data points. This procedure (which for historical reasons is called 
regression) can be applied in general to pairs of random variables. 

Computer packages are very often used to carry out the regression calculations and 
display the results. This is of special value when the data tend to follow a curve and 
various nonlinear models are tried and compared (see Section 11.5.4). 


The method of least squares 


The correlation was introduced in Section 11.4.3 as a way of measuring the dependence 
between random variables. Subsequently, we have seen how the correlation can be 
estimated and the dependence tested using sample data. We can take the idea of correla- 
tion between variables (say XY and Y) a stage further by assuming that the sample pairs 
{(X, Y;),..-, (X,, Y,,)} satisfy a linear relationship of the form 


Y=a+bX,+ EF, (=1,...,n) 


where a and b are unknown coefficients and the random variables E; have zero mean 
and represent residual errors. This assumption is prompted by the scatter diagrams 
in Figure 11.11, which illustrate how the points may be concentrated around a line. 
Figure 11.13 shows a typical scatter diagram again, this time with the line drawn in. If 
we can estimate the coefficients a and b so as to give the best fit, we shall be able to 
predict the value of Y when the value of X is known. 

The least-squares approach is to choose estimates d and 5 to minimize the sum of 
squares of the values £;; 


n 9 n . e 2 

O=> FF =Y1Y,-(4+6x)] 
i=l i=1 

Equating to zero the partial derivatives of this sum with respect to the two coefficients 

gives a pair of equations that determine the minimum: 


90 9-2 1y,- (a4 Bx, 
aw =0= 2a 0 (4+5X,)] 


dO _ _ n = i 7 
ria ll (4+ bX;)] 


These can be rewritten as 
nd + (X,X)b = (2,Y)) 
(2, X)a + (2, X7)b = (2 X,Y) 


(where »; = X’.,) from which the solution is 
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Example 11.17 


Solution 


Sey=*+¥,(X- ¥\(Y- ¥) = XV-(X)(Y) 


nN 
Sp= 4% - XP = x= (2) 
n 
S7=*E,(¥,- Py = Y= (Py 
are the sample variances and covariance. 


This process of fitting a straight line through a set of data of the form {(X, Y,),..., 
(X,,, Y,,)} is called linear regression, and the coefficients are called regression coefficients. 


A strain gauge has been bonded to a steel beam, and is being calibrated. The resistance 
of the strain gauge is converted into a voltage appearing on a meter. Known forces (X, 
in KN) are applied and voltmeter measurements (Y, in V) are as follows: 


1 2 3 4 5 6 7 8 9 10 11 12 13 14 
Y|44 49 64 7.3 88 10.3 11.7 13.2 148 15.3 16.5 17.2 189 19.3 


Fit a regression line through the data and estimate the tension in the beam when the 
meter reading is 13.8 V. 


The following quantities are calculated from the data: 


X=75, S,=403113, Y=12.0714, Sy=4.95068, XY = 110.421 
(When using a hand calculator to solve linear regression problems, it is advisable to 
work to at least five or six significant digits during intermediate calculations, because 
the subtraction in the numerator of b often results in the loss of some leading digits.) 


From these results, b = 1.22 and d= 2.89 (Figure 11.13). The estimated value of tension 
for a reading of Y = 13.8 V is given by 


13.8 = 2.89 + 1.22.X 
from which X = 8.9 kN. 


Figure 11.14 shows a pseudocode listing for linear regression. The program is very 
similar to that in Figure 11.12 for the sample correlation, and the link between these 
will be explained in Section 11.5.3. In addition to the regression coefficients @ and 5, 
an estimate of the residual standard deviation is returned, which is explained below. 
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Figure 11.14 
Pseudocode listing 
for linear regression. 
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{ Program to compute linear regression coefficients. 
x(k) and y(k) are the arrays of data, 

nis the sample size, 

xbar and ybar are the sample averages, 

sx and sy are the sample standard deviations, 

bhat is the regression slope result, 

ahat is the regression intercept result, 

se is the residual standard deviation, 

Mx, My, Qx, Qy, Qxy hold running totals. } 


Mx < 0; My — 0 
Qx + 0;Qy + 0 
Qxy — 0 
for k is | ton do 
diffx < x(k) — Mx 
diffy <— y(k) — My 
Mx < ((k — 1)*Mx+x(k))/k 
My & ((k — 1)*Myty(k))/k 
Qx & Qxt(1-1/k)*diffx*diffx 
Qy < Qyt(1-I/k)*diffy*diffy 
Qxy < Qxy+(1-1/k)*diffx«diffy 
endfor 
xbar <— Mx; ybar <— My 
sx <— sqr(Qx/n); sy — sqr(Qy/n) 
bhat <— Qxy/(n*sx*sx) 
ahat <— ybar — bhat*xbar 
se < sqr(sy*sy — bhat*bhat*sx*sx) 


Normal residuals 


The process of fitting a straight line through the data by minimizing the sum of squares 
of the errors does not involve any statistics as such. However, we often need to test 
whether the slope of the regression line is significantly different from zero, because this 
will reveal whether there is any dependence between the random variables. For this 
purpose we must make the assumption that the errors E,, called the residuals, have a 
normal distribution: 


E,~ N(O, 05) 


The unknown variance 0; can be estimated by defining 
sp=ty e=1yiy-(atéxr 
u i=1 Mt i=l 
Using the earlier result that d= Y— bX gives a more convenient form: 
1 mn Tf cae, 
Sp=-Y [(¥,- Y)-b(X%,-X 
z= S 2 [(¥,- Y) - b(X;-X)] 
= 2X (Y= 2) = 26%, -2)(- P+ BX -XY] 
i= 


= S} — 2bSyy + BS? 
= S} - PS; 
This result is used in the pseudocode listing (Figure 11.14). 
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Solution 


Various confidence intervals are derived in more advanced texts covering linear 
regression. Here the most useful results will simply be quoted. The 100(1 — a@)% 
confidence interval for the regression slope 6 is given by 


“ Se ) 
- 
(3 = Coy2,n-2 Se) =p) 


It is often useful to have an estimate of the mean value of Y for a given value of X, 
say X = x. The point estimate is d + bx, and the 100(1 — a@)% confidence interval for 
this is 


e BA Die? 
+ br troSea/ eects He 8) 3] 
= 


Estimate the residual standard deviation and find a 95% confidence interval for the 
regression slope for the data in Example 11.17. Also test the hypothesis that the tension 
in the beam is 10kN when a voltmeter reading of 15 V is obtained. 


Using the results obtained in Example 11.17, the residual standard deviation is 
S;= 0.418 
and, using 4975.1. = 2.179, the 95% confidence interval for b is 


0.418 


(1.2222.179--0438 
(4.031) 12 


)= cs, 1.29) 


Obviously the regression slope is significant — but this is not in doubt. To test the 
hypothesis that the tension is 10 kN, we can use the 95% confidence interval for the 
corresponding voltage, which is 


2 2 
299+ 122010)2:179(0818), || Ls ap /(4.031 


= (14.8, 15.4) 


The measured value of 15 V lies within this interval, so the hypothesis is accepted at 
the 5% level. A better way to approach this would be to reverse the regression (use 
force as the Y variable and voltage as the X variable), so that a confidence interval for 
the tension in the beam for a given voltage could be obtained and the assumed value 
tested. For the present data this gives (9.6, 10.1) at 95%, so again the hypothesis is 
accepted (Exercise 27). 
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Figure 11.15 
Nonlinear regression 
(Example 11.19). 
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Regression and correlation 


Both regression and correlation are statistical methods for measuring the linear depend- 
ence of one random variable upon another, so it is not surprising that there is a connection 
between them. From the definition of the sample correlation ryy (Section 11.4.4) and 
the result for the regression slope 5, it follows immediately that 


_ Suv _ Sy 


rey 


Another expression for the residual variance is then 


2 ee See Mee 2 
Sz =Sy- ie Sy=Sy(1-ryy) 
xX 


This result has an important interpretation. S} is the total variation in the Y values, and 
S; is the residual variation after the regression line has been identified, so rj, is the 
proportion of the total variation in the Y values that is accounted for by the regression 
on X: informally, it represents how closely the points are clustered about the line. This 
is a measure of goodness of fit that is especially useful when the dependence between 
X and Y is nonlinear and different models are to be compared. 


Nonlinear regression 


Sometimes the dependence between two random variables is nonlinear, and this shows 
clearly in the scatter plot; see for instance Figure 11.15. Fitting a straight line through 
the data would hardly be appropriate. Instead, various models of the dependence can be 
assumed and tested. In each case the value of rj indicates the success of the model in 
capturing the dependence. One form of nonlinear regression model involves a quadratic 
or higher-degree polynomial: 


Y,=a)+a,X,+a@X7+E, (i=1,...,n) 


The three coefficients aj, a, and a, can be identified by a multivariate regression method 
that is beyond the scope of this text. A simpler approach is to try models of the form 


Y,=aX°F, (i=1,...,n) 


P/mbar & 
1000 
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Solution 


or 
Y,=F,exp(a+bX,) (i=1,...,n) 


where each F; is a residual multiplicative error. On taking logarithms, these models 
reduce to the standard linear form: 


InY¥,=Ina+6bInX,+£, (@=1,...,n) 
or 
InY¥,=a+bX,+ FE, (i=1,...,n) 


which can be solved by the usual method. 


The following data for atmospheric pressure (P, in mbar) at various heights (H, in km) 
have been obtained: 


Height 1 0 4 8 12 16 20 
Pressure P 1012 621 286 141 104 54 
The relationship between height and pressure is believed to be of the form 

P = ecthH 


where a and b are constants. Fit and assess a model of this form and predict the atmos- 
pheric pressure at a height of 14 km. 


Taking logarithms and setting Y = In P gives 
Y,=a+bH,+ E, 
for which the following results are easily obtained: 
H=10, S,=6.83130, Y=5.43152, S,=1.01638, HY =47.4081 


Hence 


G=Y-bH=6.91 


Also, rj.y = 0.99, which implies that the fit is very good (Figure 11.15). In this case 
there is not much point in trying other models. Finally, the predicted pressure at a height 
of 14km is 


P — @5-91-0.148(14) = 126 mbar 
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11.5.5 Exercises 


Ten files of audio data are annotated by a human 
labeller. The time (s) this takes per file is a function 
of the length of the file (s) as follows: 


File length 5.4 7.9 10.0 14.2 16.1 16.8 19.6 22.0 25.0 26.7 
(x) 
Annotation 


time (Y) 


13.1 17.3 23.9 30.1 33.5 40.0 43.6 46.5 52.4 60.7 


Find the linear regression coefficients. 


Measured deflections (in mm) ofa structure under a 
load (in kg) were recorded as follows: 


Load Y 123 45 6 7 8 9 10 11 12 
Deflection Y | 16 35 45 74 86 96 106 124 134 156 164 182 


Draw a scatter plot of the data. Find the linear 
regression coefficients and predict the deflection 
for a load of 15kg. 


Using the data in Example 11.17, carry out a 
regression of force against voltage, and obtain a 
95% confidence interval for the tension in the beam 
when the voltmeter reads 15 V, as described in 
Example 11.18. 


Weekly advertising expenditures X; and sales Y; for 
a company are as follows (in units of £100): 


X,| 40 20 25 20 30 50 40 20 50 40 25 50 
Y, | 385 400 395 365 475 440 490 420 560 525 480 510 


(a) Fit a regression line and predict the sales for an 
advertising expenditure of £6000. 

(b) Estimate the residual standard deviation and find 
a 95% confidence interval for the regression 
slope. Hence test the hypothesis that the sales 
do not depend upon advertising expenditure. 

(c) Find a 95% confidence interval for the mean 
sales when advertising expenditure is £6000. 


A machine that can be run at different speeds 
produces articles, of which a certain number are 
defective. The number of defective items produced 
per hour depends upon machine speed (in rev s“') as 
indicated in the following experimental run: 


Speed 8 9 10 11 12 13 14 #15 
Defectives | 7 12 13 13 13 16 14 #18 
per hour 
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Find the regression line for number of defectives 
against speed, and a 90% confidence interval for the 
mean number of defectives per hour when the speed 
is 14revs". 


Sometimes it is required that the regression line 
passes through the origin, in which case the only 
regression coefficient is the slope of the line. 
Use the least-squares procedure to show that the 
estimate of the slope is then 


b=(LXYVUXF 
A series of measurements of voltage across and 


current through a resistor produced the following 
results: 


Voltage(V) |1 2 3 4 5 6 7 8 9 10 11 12 
Current (mA) | 6 18 27 30 42 48 58 69 74 81 94 99 


Estimate the resistance, using the result of the 
previous exercise. 


The pressure P of a gas corresponding to various 
volumes V was recorded as follows: 


50 60 70 90 100 
64.7 51.3 40.5 25.9 7.8 


V (cm*) 
P (kgcm”) 


The ideal gas law is given by the equation 
PHC 


where A and C are constants. By taking 
logarithms and using the least-squares method, 
estimate A and C from the data and predict P 
when V = 80cm’. 


The following data show the unit costs of producing 
certain electronic components and the number of 
units produced: 


50 100 250 500 1000 2000 5000 
108 65 21 13 4 2:2 1 


Lot size X; 
Unit cost Y; 


Fit a model of the form Y = aX” and predict the unit 
cost for a lot size of 300. 
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11.6.1 


Figure 11.16 
The chi-square 
distribution 
with 7,,,- 


Goodness-of-fit tests 


The common classes of distributions, especially the binomial, Poisson and normal dis- 
tributions, which often govern the data in experimental contexts, are used as the basis 
for statistical methods of estimation and testing. A question that naturally arises is whether 
or not a given set of data actually follows an assumed distribution. If it does then the 
statistical methods can be used with confidence. If not then some alternative should be 
considered. The general procedure used for testing this can also be used to test for 
dependence between two variables. 


Chi-square distribution and test 


No set of data will follow an assumed distribution exactly, but there is a general method 
for testing a distribution as a statistical hypothesis. If the hypothesis is accepted then it 
is reasonable to use the distribution as an approximation to reality. 

First the data must be partitioned into classes. If the data consist of observations 
from a discrete distribution then they will be in classes already, but it may be appropriate 
to combine some classes if the numbers of observations are small. For each class the 
number of observations that would be expected to occur under the assumed distribution 
can be worked out. The following quantity acts as a test statistic for comparing the 
observed and expected class numbers: 


where f, is the number of observations in the kth class, e, is the expected number in the 
kth class and m is the number of classes. Clearly, y* is a non-negative quantity whose 
magnitude indicates the extent of the discrepancy between the histogram of data and 
the assumed distribution. For small samples the histogram is erratic and the comparison 
invalid, but for large samples the histogram should approximate the true distribution. It 
can be shown that for a large sample the random variable y’ has a ‘chi-square’ dis- 
tribution. This class of distributions is widely used in statistics, and a typical chi-square 
probability density function is shown in Figure 11.16. We are interested in particular in 
the value of y;,,, to the right of which the area under the density function curve is @, 
where n is the (single) parameter of the distribution. These values are extensively 
tabulated; a typical table is shown in Figure 11.17. 


we 
Kean 
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n a= 0.05 a= 0.025 a=0.01 a= 0.005 n 


1 3.841 5.024 6.635 7.879 1 
2 5.991 7.378 9.210 10.597 2 
3 7.815 9.348 11.345 12.838 3 
4 9.488 11.143 13.277 14.860 4 
a 11.070 12.832 15.086 16.750 5 
6 12.592 14.449 16.812 18.548 6 
7 14.067 16.013 18.475 20.278 7 
8 15.507 17.535 20.090 21.955 8 
9 16.919 19.023 21.666 23.589 9 
10 18.307 20.483 23.209 25.188 10 
11 19.675 21.920 24.725 26.757 11 
12 21.026 23.337 26.217 28.300 12 
13 22.362 24.736 27.688 29.819 13 
14 23.685 26.119 29.141 31.319 14 
15 24.996 27.488 30.578 32.801 15 
16 26.296 28.845 32.000 34.267 16 
17 27.587 30.191 33.409 35.718 17 
18 28.869 31.526 34.805 37.156 18 
19 30.144 32.852 36.191 38.582 19 
20 31.410 34.170 37.566 39.997 20 
21 32.671 35.479 38.932 41.401 21 
22 33.924 36.781 40.289 42.796 22 
23 35.172 38.076 41.638 44.181 23 
24 36.415 39.364 42.980 45.558 24 
25 37.652 40.646 44.314 46.928 25 
26 38.885 41.923 45.642 48.290 26 
27 40.113 43.194 46.963 49.645 27 
28 41.337 44.461 48.278 50.993 28 
29 42.557 45.722 49.588 52.336 29 
30 43.773 46.979 50.892 53.672 30 


The hypothesis of the assumed distribution is rejected if 


2 2 
Xx > Xo,m-t-1 


where @ is the significance level and ¢ is the number of independent parameters esti- 
mated from the data and used for computing the e, values. The significance level is the 
probability of false rejection, as discussed in Section 11.3.4. The only difference is that 
there is now no estimation procedure underlying the hypothesis test, which must stand 
alone. Sometimes the hypothesis is deliberately vague, for example a parameter value 
may be left unspecified. If the data themselves are used to fix parameter values in the 
assumed distribution before testing then the test must be strengthened to allow for this 
in the form of a correction ¢ in the chi-square parameter. 

A useful rule of thumb when using this test is that there should be at most a small 
number (one or two) of classes with an expected number of observations less than five. 
If necessary, classes in the tails of the distribution can be merged. 
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Solution 


Example 11.21 


Solution 


Figure 11.18 
Chi-square calculation 
for Example 11.21. 


A die is tossed 600 times, and the numbers of occurrences of the numbers one to six are 
recorded respectively as 89, 113, 98, 104, 117 and 79. Is the die fair or biased? 


The expected values are e, = 100 for all k, and the test value is y* = 10.4. This is less 
than 759s; = 11.07, so we should expect results as erratic as this at least once in 20 
similar experiments. The die may well be biased, but the data are insufficient to justify 
this conclusion. 


The numbers of trucks arriving per hour at a warehouse are counted for each of 500h. 
Counts of zero up to eight arrivals are recorded on respectively 52, 151, 130, 102, 45, 
12,5, 1 and 2 occasions. Test the hypothesis that the numbers of arrivals have a Poisson 
distribution, and estimate how often there will be nine or more arrivals in one hour. 


The hypothesis stipulates a Poisson distribution, but without specifying the parameter 
A. Since the mean of the Poisson distribution is 7 and the average number of arrivals 
per hour is 2.02 from the data, it is reasonable to assume that A = 2. The columns in the 
table in Figure 11.18 show the observed counts f,, the Poisson probabilities p,, the 
expected counts e, = 500 p, and the individual y’ values for each class. The last two 
classes have been combined because the numbers are so small. One parameter has 
been estimated from the data, so the total 7° value is compared with 759;4= 12.59. The 
Poisson hypothesis is accepted, and on that basis the probability of nine or more trucks 
arriving in one hour is 


8 k -2 


P(9 or more) = 1- ze = 0.000 237 
k=0 


This will occur roughly once in every 4200 h of operation. 


Trucks i Py & x 

0 52 0.1353 67.7 3.63 
1 151 0.2707 135.3 1.81 
2 130 0.2707 135.3 0.21 
3 102 0.1804 90.2 1.54 
4 45 0.0902 45.1 0.00 
5 12 0.0361 18.0 2.02 
6 5 0.0120 6.0 0.17 
7 3 0.0046 23 0.24 


Totals 500 1.0 500 9.62 


Because so many statistical methods assume normal data, it is important to have a test 
for normality, and the chi-square method can be used (Exercise 38 and Section 11.8.4). 
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Figure 11.19 
Contingency table. 
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Contingency tables 


In Section 11.4.3 the correlation was introduced as a measure of dependence between 
two random variables. The sample correlation (Section 11.4.4) provides an estimate 
from data. This measure only applies to numerical random variables, and then only 
works for linear dependence (Exercise 15). The rank correlation (Section 11.4.6) has 
more general applicability, but still requires that the data be classified in order of rank. 
The chi-square testing procedure can be adapted to provide at least an indicator of 
dependence that has the widest applicability of all. 

Suppose that each item in a sample of size n can be separately classified as one of 


A,,...,A, by one criterion, and as one of B,,..., B. by another (these may be numer- 
ical values, but not necessarily). The number /;, of items in the sample that are classified 
as ‘A; and B, can be counted for eachi=1,...,randj=1,...,c. The table of these 


numbers (with r rows and c columns) is called a contingency table (Figure 11.19). The 
question is whether the two criteria are independent. If not then some combinations of 
A, and B, will occur significantly more often (and others less often) than would be 
expected under the assumption of independence. We first have to work out how many 
would be expected under an assumption of independence. 


1 
Class | B, Ms B, | Total 
t 1 
le 
{ 
I I 
| 
I 
| 
eee 
[ 

A, In e Tee tis 
| 

Total | fi, |. fey n 


Let the row and column totals be denoted by 


fea hi (i=1,...,n 


hash, (VGG=1,...,¢) 


If the criteria are independent then the joint probability for each combination can be 
expressed as the product of the separate marginal probabilities: 


P(A, M B) = P(A) P(B)) 


The chi-square procedure will be used to see how well the data fit this assumption. To 
test it, we can estimate the marginal probabilities from the row and column totals, 


p=, PB) = 4 
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Example 11.22 


Figure 11.20 
Contingency table 
for Example 11.22. 


and multiply the product of these by n to obtain the expected number e, for each 
combination: 


e= pts bi = Fi Sri 
: nn n 


The chi-square goodness-of-fit statistic follows from the actual and expected numbers 
(f;; and e,) as a sum over all the rows and columns: 


r c Seabee 2 
¥¢ = y y hii a 


i=l j=l 


If the value of this is large then the hypothesis of independence is rejected, because 
the actual and expected counts differ by more than can be attributed to chance. As 
explained in Section 11.6.1, the largeness is judged with respect to ¥2,,,,, from the 
chi-square table. The number of classes, m, is the number of rows times the number of 
columns, rc. The number of independent parameters estimated from the data, f, is the 
number of independent marginal probabilities P(4;) and P(B)): 


t=(r-—1)+(c-1) 


The number is not 7 + c, because the row and column totals must equal one, so when 
all but one are specified, the last is determined. Finally, 


m—-t-l=re-(r+ec-—2)-1=(r-1)(c-1) 


The hypothesis of independence is therefore rejected (at significance level q) if 


2 2 
I 2 eGo: 


An accident inspector makes spot checks on working practices during visits to indus- 
trial sites chosen at random. At one large construction site the numbers of accidents 
occurring per week were counted for a period of three years, and each week was also 
classified as to whether or not the inspector had visited the site during the previous 
week. The results are shown in bold print in Figure 11.20. Do visits by the inspector 
tend to reduce the number of accidents, at least in the short term? 


Number of accidents 
0 1 2 3 Total 
Visit 20 (13.38) 3 (7.08) 1 (2.46) 0 (1.08) 24 
Residual 2.96 -1.99 —1.07 —1.16 
No visit 67 (73.62) 43 (38.92) 15 (13.54) 7 (5.92) 132 
Residual 2.96 1.99 1.07 116 
Total 87 46 16 7 156 
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Solution The respective row and column totals are shown in Figure 11.20, together with the 
expected numbers e,, in parentheses in each cell. For example, the top left cell has 
observed number 20, row total 24, column total 87, m = 156, and hence the expected 
number 


€,, = (24)(87)/156 = 13.38 


The chi-square sum is 


2. 2 
p= 20 13.38) a + 2-592) 


= 8.94 


With two rows, four columns and a significance level of 5%, the appropriate number 
from the chi-square table is 759s; = 7.815. The calculated value exceeds this, and by 
comparing the observed and expected numbers in the table, it seems clear that the visits 
by the inspector do tend to reduce the number of accidents. This is not, however, 
significant at the 2.5% level. 


A significant chi-square value does not by itself reveal what part or parts of the table 
are responsible for the lack of independence. A procedure that is often helpful in this 
respect is to work out the adjusted residual for each cell, defined as 


d.= jee Cif 
7 el —f./r)C —fy/n)] 


Under the assumption of independence, these are approximately standard normal, so a 
significant value for a cell suggests that that cell is partly responsible for the depend- 
ence overall. The adjusted residuals for the contingency table in Example 11.22 are 
shown in Figure 11.20, and support the conclusion that visits by the inspector tend to 
reduce the number of accidents. 

For a useful survey of procedures for analysing contingency tables see B. S. Everitt, 
The Analysis of Contingency Tables, 2nd edn, Chapman & Hall, London, 1992. 


11.6.3 Exercises 


In a genetic experiment, outcome A is expected to 35 — The number of books borrowed from a library 
occur twice as often as outcome B, which in turn is that is open five days a week is as follows: 
expected to occur twice as often as outcome C, and Monday 153, Tuesday 108, Wednesday 
exactly one of these three outcomes must occur. In 120, Thursday 114, Friday 145. Test (at 

a sample of size 100, outcomes A, B, C occurred 5% significance) whether the number of 

63, 22, 15 times respectively. Test the hypothesis at books borrowed depends on the day of 

5% significance. the week. 
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Si 


38 


ay 


A new process for manufacturing light fibres is 

being tested. Out of 50 samples, 32 contained no 

flaws, 12 contained one flaw and 6 contained two 

flaws. Test the hypothesis that the number of flaws 40 
per sample has a Poisson distribution. 


In an early experiment on the emission of 
a-particles from a radioactive source, Rutherford 
obtained the following data on counts of particles 
during constant time intervals: 


Number of 
particles 

Number of | 57 203 383 525 532 408 273 139 45 27 10 6 
intervals 


Test the hypothesis that the number of particles 
emitted during an interval has a Poisson 
distribution. 


Two samples of 100 data have been grouped into 

classes as shown in Figure 11.21. The sample 

average and standard deviation in each case 

were 10.0 and 2.0 respectively. 4l 


(a) Draw the histogram for each sample. 
(b) Test each sample for normality using the 
measured parameters. 


Class Sample 1 Sample 2 
<6.5 4 3 
6.5-7.5 6 6 
7.5-8.5 16 16 
8.5-9.5 16 13 
9.5-10.5 17 26 
10.5—11.5 20 7 
11.5-12.5 12 19 
12.5-13.5 6 > 
>13.5 3 5 


Figure 11.21 Data classification for 
Exercise 38. 42 


Shipments of electronic devices have been 
received by a firm from three sources: A, B 

and C. Each device is classified as either 

perfect, intermediate (imperfect but acceptable), 
or unacceptable. From source A 89 were 

perfect, 23 intermediate and 12 unacceptable. 
Corresponding figures for source B were 62, 12 
and 8 respectively, and for source C 119, 30 and 
21 respectively. Is there any significant difference 


in quality between the devices received from the 
three sources? 


Cars produced at a factory are chosen at random 
for a thorough inspection. The number inspected 
and the number of those that were found to be 
unsuitable for shipment were counted monthly 
for one year as follows: 


Month Jan. Feb. Mar. Apr. May Jun. 


Inspected | 450 550 550 400 600 450 
Defective 8 14 6 3 7 8 


Month Jul. Aug. Sep. Oct. 


Inspected | 450 200 450 600 600 550 
Defective 16 5 12 6 15 9 


Is there a significant variation in quality through 
the year? 


Customers ordering regularly from an on-line 
clothing catalogue are classed as low, medium and 
high spenders. Considering four products from the 
catalogue (a jacket, a shirt, a pair of trousers, and a 
pair of shoes), the numbers of customers in each 
class buying these products over a fixed period of 
time are given in the following table: 


Spending level | Jacket | Shirt | Trousers | Shoes 
Low oA 94 57 113 
Medium 66 157 94 209 
High 58 120 41 125 


Does this table provide evidence that customers 
with different spending levels tend to choose 
different products? 


A quality control engineer takes daily samples of 
four television sets coming off an assembly line. 
In a total of 200 working days he found that on 
110 days no sets required adjustments, on 73 days 
one set requires adjustments, on 16 days two sets 
and on | day three sets. Use these results to test 
the hypothesis that 10% of sets coming off the 
assembly line required adjustments, at 5% and 1% 
significance levels. Also test this using the 
confidence interval for proportion (Section 11.3.6), 
using the total number of sets requiring adjustments. 
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Moment generating functions 


This section is more difficult than the rest of this chapter, and can be treated as optional 
during a first reading. It contains the proofs of theoretical results, such as the central 
limit theorem, that are of great importance, as seen earlier in this chapter. The technique 
introduced here is the moment generating function, which is a useful tool for finding 
means and variances of random variables as well as for proving these essential results. 
The moment generating function also bears a striking resemblance to the Laplace trans- 
form considered in Chapter 5. 


Definition and simple applications 


The moment generating function of a random variable X is defined as 


»y e'"P(X=v,) _ inthe discrete case 
My(t) = E(e*) = 4 
e” fy (x) dx in the continuous case 


-0o 


where ¢ is a real variable. This is an example of the expected value of a function h(X) 
of the random variable X (Section 11.2.2). The moment generating function does not 
always exist, or it may exist only for certain values of ¢. In cases where it fails to exist 
there is an alternative (called the characteristic function) which always exists and has 
similar properties. When the moment generating function does exist, it is unique in the 
sense that no two distinct distributions can have the same moment generating function. 

To see how the moment generating function earns its name, the first step is to differ- 
entiate it with respect to ¢ and then let ¢ tend to zero: 


£ MC) leo = E(X e*) |g = E(X) = fy 


Thus the first derivative gives the mean. Differentiating again gives 
a 

Ss My(t) 0 = EX? e*) 0 = EX”) 

dt 

From this result we obtain the variance (from Section 11.2.2) 


Var(X) = E(X’) — [E(X)P 


We can summarize these results as follows. If a random variable X has mean Uy, 
variance 0; and moment generating function M,(f) defined for ¢ in some neighbour- 
hood of zero then 


2 
My=MY(0), oy =(MY'(0) - [My(0)T) 
where the superscript in parentheses denotes the order of the derivative. Furthermore, 


E(X")=M(0) k=(1,2,...) 
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Example 11.23 — Show that the mean and variance of the Poisson distribution with parameter A are both 
equal to A (Section 11.2.3). 


Solution IfX has a Poisson distribution then 
(k=0,1,2,...) 
and the moment generating function is 


oy k -A’ co ty\k 
My(t) = ye = vy acy = exp[A(e'- 1)] 
! ak! 


k=0 
Differentiating this with respect to ¢ gives 
M(t) = Ae'exp[A(e'- 1)] 
MY (t) =(K e* +A )exp[A(e'- 1)] 
from which 
My=My’(0)=A 
Ox=My(0)-py=A 


as expected. 


Example 11.24 — Show that the mean and standard deviation of the exponential distribution 


Ae™ (x= 0) 


Sx(x) -| 
0 (x < 0) 


are both equal to 1/A (Section 11.10.2). 


Solution The moment generating function is 


ie oe 
uso= | ae dx= > 


Note that the integral only exists for ¢ < A, but we can differentiate it and set t = 0 
for any positive value of A: 


Miy'(0) = 2" = My 
MY (0) = 20° 


from which o, = 2°. 
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11.7.2 The Poisson approximation to the binomial 


In addition to its utility for finding means and variances, the moment generating func- 
tion is a very useful theoretical tool. In this section and the next we shall use it to prove 
two of the most important results in probability theory, but first we need the following 
general property of moment generating functions. 


Suppose that X and Y are independent random variables with moment generating 
functions M,(t) and M,(t). Then the moment generating function of their sum is 
given by 


My.y(t) = My(QMy (1) 


To prove this, we shall assume that XY and Y are continuous random variables with a 
joint density function fyy(x, y); however, the proof is essentially the same if either or 
both are discrete. By definition, 


My,y(t) = Ele” ] = | | el Ff y(x, y) dxdy 


Both factors of the integrand themselves factorize (noting the independence of X and 
Y) to give 


My,y(t) = | | le" f-(x) le") dxdy 
The two integrals can now be separated, and the result follows: 


Mierid= | c“hionax | e” fy (y) dy 


= My(x)My(y) 
It follows that if {X,, ..., X,,} are independent and identically distributed random 
variables, each with moment generating function M,(f), and if Z=X,+...+X,, 


then 
M,(t) = [My(o]" 


We now proceed to the main result. If the random variable Y has a binomial distribu- 
tion with parameters n and p and the random variable X has a Poisson distribution 
with parameter A = np then as n > ~ and p — 0, the distribution of X tends to that 
One 


First let the random variable B have a Bernoulli distribution with parameter p 
(Section 11.2.3). The moment generating function of B is then 
M,(t) = ep + (1- p)=1+ple'- 1) 


Since the binomial random variable Y is the sum of n copies of B, it follows that the 
moment generating function of the binomial distribution is 


My(t) = [1 + p(e’— DI 
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11.7.3 


Theorem 11.1 


Proof 


It can be proved that for any real z 


lim (1 + 2) =e 
noo n 
Now let z = np(e’ — 1) and assume that A = np: 
M,(t) = (1 +2) ~ e' = exp[A(e'- 1)] = My(2) 
n 


(the moment generating function for the Poisson distribution was derived in Example 
11.23). The uniqueness of the moment generating function implies that the distributions 
of X and Y must be similar, provided that n is sufficiently large. It is also required that 
Pp — 0asn—-, so that z does not grow without bound. 

This approximation has many applications; see for example Section 11.9.2. 


Proof of the central limit theorem 


This theorem is of vital importance in statistics because it tells us that the sample aver- 
age for a sample of at least moderate size tends to have a normal distribution even when 
the data themselves do not. The result (Section 11.2.4) is here restated and proved using 
the moment generating function. 


The central limit theorem 


If {X,, ..., X,} are independent and identically distributed random variables, each 
with mean jl, and variance oj, and if 


AG So on tO = Ae 
On 


then the distribution of Z, tends to the standard normal distribution as n > -~. 


, 


n 


First let the random variables Y, be defined by 
aa (i=1,...,7) 


so that 
E(Y,) =0 
AY?) =4 
n 
E(Y;})= . , where c is a constant 


Expanding the moment generating function M,(f) for each Y,; as a Maclaurin series (to 
four terms) gives 


2 3 
My(t) ~ M,(0)+ MP (O)t+ MP0) = + MV(0) 5 
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By the results in Section 11.7.1, the coefficients of this series can be replaced by the 


successive moments E(Y*): 


2, 3 
M,(t) = EI) + E(t + BV) 5 + BO) = 1+ 


Now Z, is the sum of the variables Y,, 


Z= DY, 
i=1 


t ct 


2n 6n\n 


so the moment generating function of Z, is the nth power of that of Y;: 


Mz,(t) = [My(0)]" 


Retaining only the first two terms of a binomial expansion of this gives 


Mz (t) = (1 + +) # nf +. 


n 2n 


/2 


2 
se” asn oo 


2\n-1 ra 
| ean 
6n\n 


because all terms except the first will tend to zero. 
It only remains to show that this is the moment generating function for the standard 


normal distribution; see Exercise 47. 


11.7.4 Exercises 


A continuous random variable X has density 
function 


0 


(x = 0) 

46 
Find the value of the constant c. Derive the moment 
generating function, and hence find the mean and 
variance of X. 


Prove that if X;, . . 
random variables with parameters A), . . 
their sum 


., X, are independent Poisson 
., A, then 


Y=X,+...+X, 


is also Poisson, with parameter 1 =A, +...+A.,,. 


A factory contains 30 machines each with 

breakdown probability 0.01 in any one hour and 47 
40 machines each with breakdown probability 

0.005 in any one hour. Use the result of Exercise 44, 
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end of theorem 


together with the Poisson approximation to the 
binomial, to find the probability that a total of 
three or more machine breakdowns will occur 
in any one hour. 


A manufacturer has agreed to dispatch small 
servomechanisms in cartons of 100 to a distributor. 
The distributor requires that 90% of cartons contain 
at most one defective servomechanism. Assuming 
the Poisson approximation to the binomial, obtain 
an equation for the Poisson parameter A such that 
the distributor’s requirements are just satisfied. 
Solve for A by one of the standard methods for 
nonlinear equations (approximate solution 0.5), 
and hence find the required proportion of 
manufactured servomechanisms that must be 
satisfactory. 


Show that the moment generating function 
of the standard normal distribution is e"” 
(Hint: Complete the square in the exponential.) 
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11.8.1 


Figure 11.22 Data for 
engine case study. 


performance data 


Introduction 


Statistical methods are often used in conjunction with each other. So far in this chapter 
the examples and exercises have almost always been designed to illustrate the various 
topics one at a time. This section contains an example of a more extended problem to 
which several topics are relevant, and correspondingly there are several stages to the 
analysis. 

The background to the problem is this. Suppose that the fuel consumption of a car 
engine is tested by measuring the time that the engine runs at constant speed on a litre 
of standard fuel. Two prototype engines, A and B, are being compared for fuel con- 
sumption. For each engine a series of tests is performed in various ambient temperatures, 
which are also recorded. Figure 11.22 contains the data. There are 30 observations each 
for the four random variables concerned: 


A, running time in minutes for engine A; 

T, ambient temperature in degrees Celsius for engine A; 

B, running time in minutes for engine B; 

U, ambient temperature in degrees Celsius for engine B. 
The histograms for the running times are compared in Figure 11.23(a) and those for the 
temperatures in Figure 11.23(b). The overall profile of temperatures is very similar for 
the two series of tests, differing only in the number of unusually high or low figures 
encountered. The profiles of running times appear to differ rather more markedly. It is 


clear that displaying the data in this way is useful, but some analysis will have to be 
done in order to determine whether the differences are significant. 


Engine A Engine B 
A £ A T B U B U 
2737 24 24.1 i 24.9 13 24.3 17 
24.3 25 23:1 14 21.4 19 24.5 16 
23.7 18 23.4 16 24.1 18 26.1 18 
22.1 15 23.1 9 27:5 19 20-7, 14 
21.8 19 24.1 14 215 21 24.3 19 
24.7 16 28.6 23 25.7 17 26.1 5 
23.4 17 20.2 14 24.9 17 24.0 17 
21.6 14 25.7 18 2373 19 24.9 18 
24.5 18 24.6 18 22.5 21 26.7 23 
26.1 20 24.0 12 28.5 12 27.3 28 
24.8 15 24.9 18 25.9 17 23.9 18 
23.7 15 21.9 20 26.9 13 23.1 10 
25.0 22 25.1 16 217 17 25:5 25 
26.9 18 25.7 16 25.4 23 24.9 22 
23.7 19 23.5 11 25.3 30 25:9 16 
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Figure 11.23 
Histograms of engine 
data: (a) running times; 
(b) temperatures. 
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Number 


ODEN WAMNA~A1I MO OO 


Number [a] 


8 
7 
6 
5 
4 
3 
2 cae 
! { 
1 i) ts 
0 — 
5 10 12.5 15 17.5 20 225 25 275 30 
(b) Ambient temperature/°C 


When planning the analysis, it is as well to consider the questions to which most 
interest attaches. Do the mean running times for the two engines differ? Does the run- 
ning time depend on temperature? If so, and if there is a difference in the temperatures 
for the test series on engines A and B, can this account for any apparent difference in 
fuel consumption? More particularly, are the data normal? This has a bearing on the 
methods used, and hence on the conclusions drawn. 


Difference in mean running times and temperatures 


The sample averages and both versions of standard deviation for the data in Figure 11.22 
are as follows: 


4=2420, T=16.70, 8=25.36, UW=18.07 
S,=1.761, S,=4.001, S,=1.657, S,=4.932 
Siy1= 1.791, Spy = 4.070, Sp, = 1.685, Sy, = 5.017 


The average running time for engine B is slightly higher than for engine A. The sample 
standard deviations are very similar, so we can assume that the true standard deviations 
are equal and use the method for comparing means discussed in Section 11.3.5. The 
pooled estimate of the variance is 


52 = DS + Soni) _ 3.208 + 2.839 


i = 3.023 
2(n- 1) 2 


www.z2Ofile.org 


960 APPLIED PROBABILITY AND STATISTICS 


11.8.3 


and the relevant value from the f-table is f) 9955, = 1.960. In fact the sample is large 
enough for the value for the normal distribution to be taken. The 95% confidence 
interval for the difference U, — LL, is 


(B — A + 1.96S,/15) = (0.28, 2.04) 


Even the 99% confidence interval (0.12, 2.20) does not contain the zero point, so we 
can be almost certain that the difference in mean running times is significant. 

Following the same procedure for the temperatures gives the 95% confidence inter- 
val for the difference 1, — 1; to be (—0.94, 3.68). Superficially, this is not significant — 
and even if the running times do depend on temperature, the similarity of the two test 
series in this respect enables this factor to be discounted. If so, the fuel performance for 
engine B is superior to that for engine A. However, if the temperature sensitivity were 
very high then even a difference in the average too small to give a significant result by 
this method could create a misleading difference in the fuel consumption figures. This 
possibility needs to be examined. 


Dependence of running time on temperature 


The simplest way to test for dependence is to correlate times and temperatures for each 
engine. To compute the sample correlations, we need the following additional results 
from the data: 


AT = 407.28, | BU = 457.87 


The sample correlations (Section 11.4.4) of A and 7 and of B and U are then 


ieee OD as, ee BE 9030 
SSr ; SpSy 


and the respective 95% confidence intervals (Section 11.4.5) are 
(0.10, 0.69), (—0.39, 0.34) 


This is a quite definitive result: the running time for engine A depends positively upon 
the ambient temperature, but that for engine B does not. The confidence intervals are 
based on the assumption that all the variables A, 7, B and U are normal. The histograms 
have this character, and a test for normality will be covered later. 

Linear regression also reveals the dependence of running time on temperature. Here 
we assume that the variables are related by a linear model as follows: 


A;=ey+d,T, +E; ‘ 
(i=1,...,7”) 


B,= Cpt+d,U,+F; 
where c,, d,, Cc, and d, are constants, and the random variables E, and F, represent 


residual errors. Using the results of the least-squares analysis in Section 11.5.1, for 
engine A we have 


d,=-4 OO _019,  é,=4-d,7=209 
T 
Likewise, for engine B 


d, =—0.010, Cy = 25.5 
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Time/min 
30 


+ = t t 1 20 
10 15 20 25 30 0 ) 10 15 20 25 30 
Temperature/°C Temperature/°C 


(a) (b) 


Figure 11.24 Regression of running time against temperature: (a) engine A; (b) engine B. 


Figure 11.24 contains scatter plots of the data with these regression lines drawn. The 
points are well scattered about the lines. The residual variances, using the results in 
Sections 11.5.2 and 11.5.3, are 


S2=S2=¢,82 38.0 =77,) 2249 
S?.=2.74 


As explained in Section 11.5.3, the respective values of r? indicate the extent to which the 
variation in running times is due to the dependence on temperature. For engine B there 
is virtually no such dependence. For engine A we have r7,7 = 0.198, so nearly 20% of 
the variation in running times is accounted for in this way. 

If we assume that the residuals £; and F; are normal, we can obtain confidence 
intervals for the regression slopes. The appropriate value from the ¢ table is fy. ,-. = 
to.025,28 = 2.048, so the 95% confidence interval for d, is 


S 


TV 


(a + 2.048 } = (0.04, 0.35) 
The significance shown here confirms that found for the correlation. The 95% confidence 
interval for d, is (—0.14, 0.12). 

We now return to the main question. We know that the average running time for 
engine A was significantly lower than that for engine B. However, we also know that 
the running time for engine A depends on temperature, and that the average temperature 
during the test series for engine A was somewhat lower than for engine B. Could this 
account for the difference? 

A simple way to test this is to look at the average values. The difference in average 
temperatures is (18.07 — 16.70) = 1.37 and the regression slope d, is estimated to be 
0.196, so that a deficit in the average running time of (1.37)(0.196) = 0.27 min is 
predicted on this basis. This cannot account for the actual difference of 1.16 min, and 
does not even bring the zero point within the 95% confidence interval for the difference 
in means LM, — Ly. 

To examine this more carefully, we can try using the regression slopes to correct all 
running times to the same temperature. Choosing the average temperature for the series 
B for this purpose, we can let 
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Figure 11.25 
Predicted difference in 
mean running times. 


11.8.4 


Difference/min 


~ 95% confidence limits 


~ 


Temperature/°C 


4 


X,=A,+d,(0 - T,) 
Y,=B,+d,(U - U;) 


The 95% confidence interval for the difference between mean running times, [ly — Hy, 
at this temperature is then (0.06, 1.72), using the method in Section 11.3.5. The problem 
with this is that the estimates of d, and d, themselves have rather wide confidence 
intervals, and it is not satisfactory to adopt a point value to apply the temperature 
correction. 

We need a more direct way to obtain the confidence interval for the difference 
between mean running times at any particular temperature. This is possible using a 
more general theory of linear regression, formulated using matrix algebra, which allows 
for any number of regression coefficients instead of just two; see for example G. A. F. 
Seber, Linear Regression Analysis (Wiley, New York, 1977). Applied to the present 
problem, estimates of the regression slopes d, and d, and intercepts c, and c, are obtained 
simultaneously, with the same values as before, and it now becomes possible to obtain 
a confidence interval for any linear combination of these four unknowns. In particular, 
the confidence interval for the difference between mean running times at any temperature 
t is based on the linear combination 


(cy + dgt) — (cy + dit) 


Space precludes coverage of the analysis here, but the results can be seen in Figure 11.25. 
At any temperature below 22.4°C engine B is predicted to have the advantage over 
engine A (this temperature being the point at which the regression lines cross). At any 
temperature below 18.1°C the 95% confidence interval for the difference is entirely 
positive, and we should say that engine B has a significant advantage. This is the best 
comparison of the engines that is possible using the data presented. 


Test for normality 


The confidence interval statistics are all based on the assumption of normality of the 
data. Although the sample sizes are reasonably large, so that the central limit theorem 
can be relied upon to weaken this requirement, it is worth applying a test for normality 
to see whether there is any clear evidence to the contrary. Here the regression residuals 
E, and F;, will be tested using the method described in Section 11.6.1. 

Figure 11.26 shows the histogram of all 60 ‘standardized’ residuals. The residuals 
have zero mean in any case, and are standardized by dividing by the standard deviation 
so that they can be compared with a standard normal distribution. It is convenient to use 
intervals of width 0.4, and the comparison is developed in Figure 11.27. 
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Figure 11.26 
Histogram of residuals. 


Figure 11.27 Table of 
the test for normality. 


11.8.5 


12 
10 
8 
6 
4 
A A | 
oth 

~2.4 -2.0 -1.6 -12 -08-04 0 04 08 12 16 20 24 
Interval Observed ( f,) Probability Expected (e,) Chi-square 
(—ee, —1.4) 5 0.0808 4.848 0.005 
(—1.4, -1.0) 4 0.0779 4.674 0.097 
(—1.0, —0.6) 6 0.1156 6.936 0.126 
(—0.6, —0.2) 10 0.1464 8.784 0.168 
(—0.2, +0.2) 8 0.1586 9.516 0.242 
(+0.2, +0.6) 11 0.1464 8.784 0.559 
(+0.6, +1.0) 5 0.1156 6.936 0.540 
(+1.0, +1.4) 8 0.0779 4.674 2.367 
(+1.4, +e0) 3 0.0808 4.848 0.704 
Totals 60 1.0 60 4.809 


The normal probabilities for each interval are obtained from the standard normal table 
of the cumulative distribution function ®(z), Figure 11.2, taking successive differences: 


P@, < Z< Z) = O(Z,) — O(,) 


These probabilities are multiplied by 60 to obtain the expected number in each interval, 
and the difference between the observed and expected number for each interval is 
squared and then divided by the expected number to give the contribution to the total 
chi-square: 


; m cn _ ex) 
=y— =4381 
a I ae 
k=1 
This is small compared with 76 s,s = 15.507, so the hypothesis of normality is accepted. 
It is unwise when applying this test in general to have many classes with expected 
numbers less than five, so the intervals in the tails of the histogram have been merged. 


Conclusions 


All the questions posed in Section 11.8.1 have now been answered. Engine B has an 
average running time that is significantly higher than that for engine A, showing that it 
has the advantage in fuel consumption. However, this statement requires qualification. 
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The running time for engine A depends upon ambient temperature. The temperature 
difference between the two test series was not significant, and does not account for the 
difference in average running times. However, engine B will only maintain its fuel 
advantage up to a certain point. This point has been estimated, but cannot be identified 
very precisely because of considerable residual scatter in the data. There are many 
potential sources of this scatter, such as errors in measuring out the fuel, or variations 
in the quantity and consistency of the engine oil. The scatter has a normal distribution, 
which justifies the statistics behind the conclusions reached. 


ee aediiccalicalyseciienm statistical quality control 


11.9.1 Introduction 


Every manufacturer recognizes the importance of quality, and every manufacturing pro- 
cess involves some variation in the quality of its output, however that is to be measured. 
Experience shows that tolerating a lack of quality tends to be more costly in the 
end than promoting a quality approach. It follows that quality control is a major and 
increasing concern, and methods of statistical quality control are more important than 
ever. The domain of these methods now extends to the construction and service indus- 
tries as well as to manufacturing — wherever there is a process that can be monitored in 
quantitative terms. 

Traditionally, quality control involved the accumulation of batches of manufactured 
items, the testing of samples extracted from these batches, and the acceptance or rejec- 
tion (with appropriate rectifying action) of these batches depending upon the outcome. 
The essential problem with this is that it is too late within the process: it is impossible 
to inspect or test quality into a product. More recently the main concern has been to 
design the quality into the product or service and to monitor the process to ensure that 
the standard is maintained, in order to prevent any deficiency. Assurance can then be 
formally given to the customer that proper procedures are in place. 

Control charts play an important role in the implementation of quality. The idea of 
these is introduced in Section 13.6 in Modern Engineering Mathematics, where Shewhart 
charts for counts of defectives are described. In order for this section to be as self- 
contained as possible, some of that material is repeated here. This section then covers 
more powerful control charts and extends the scope of what they monitor. 

First note that there are two main alternative measures of quality: attribute and 
variable. In attribute measure, regular samples from the process are inspected and for 
each sample the number that fail according to some criterion is plotted on a chart. In 
variable measure, regular samples are again taken, but this time the sample average for 
some numerical measure (such as dimension or lifetime) is plotted. 


11.9.2 Shewhart attribute control charts 


Figure 11.28 is an example of a Shewhart control chart: a plot of the count of ‘defectives’ 
(the number in the sample failing according to some chosen criterion) against sample 
number. It is assumed that a small (specified) proportion of ‘defective’ items in the 
process is permitted. Also shown on the chart are two limits on the count of defectives, 
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Attribute control chart 
for Example 11.25. 


Example 11.25 
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10 
Action limit 


Count of defectives 


0 2 4 6 8 10 12 14 16 18 20 
Sample number 


corresponding to probabilities of one in 40 and one in 1000 of a sample count falling 
outside if the process is ‘in control’; that is, conforming to the specification. These are 
called warning and action limits respectively, and are denoted by cy and c,. 

Any sample point falling outside the action limit would normally result in the pro- 
cess being suspended and the problem corrected. Roughly one in 40 sample points will 
fall outside the warning limit purely by chance, but if this occurs repeatedly or if there 
is a clear trend upwards in the counts of defectives then action may well be taken before 
the action limit itself is crossed. 

To obtain the warning and action limits, we use the Poisson approximation to the 
binomial. If the acceptable proportion of defective items is p, usually small, and the 
sample size is n then for a process in control the defective count C, say, will be a 
binomial random variable with parameters n and p. Provided that n is not too small, the 
Poisson approximation can be used (Section 11.7.2): 


oo k —np 
P(C & o) = y SPS 
- ! 


1000 
limit cy and the action limit c, respectively, in terms of the product np. This is the basis 


for the table shown in Figure 11.29, which enables cy and c, to be read directly from 
the value of np. 


Equating this to { and then to 4; gives equations that can be solved for the warning 


Regular samples of 50 are taken from a process making electronic components, for 
which an acceptable proportion of defectives is 5%. Successive counts of defectives in 
each sample are as follows: 


7 8 9 10 11 12 13 #14 «15 16 #17 «+18 #19 20 
442 6 7 4 5 5 8 6 5 9 7 8 


At what point would the decision be taken to stop and correct the process? 
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Solution 


Figure 11.29 
Shewhart attribute 
control limits: 7 is 
sample size, p is 
probability of defect, 
Cw is warning limit and 
Ca is action limit. 


The control chart is shown in Figure 11.28. From np = 2.5 and Figure 11.29 we have the 
warning limit cy = 5.5 and the action limit c, = 8.5. The half-integer values are to avoid 
ambiguity when the count lies on a limit. There are warnings at samples 6, 10, 11, 15 and 
16 before the action limit is crossed at sample 18. Strictly, the decision should be taken 
at that point, but the probability of two consecutive warnings is less than one in 1600 
by the product rule of probabilities, which would justify taking action after sample 11. 


Cw OF Cy np for cw np for cy, 
135 <0.44 <0.13 
25 0.44—0.87 0.13-0.32 
35 0.87-1.38 0.32—0.60 
4.5 1.38-1.94 0.60—0.94 
5:5 1.94—2.53 0.94—1.33 
6.5 2.53-3.16 1.33-1.77 
7.5 3.16-3.81 1.77-2.23 
8.5 3.81—4.48 2.23-2.73 
95 4.48—5.17 2.73-3.25 
10.5 5.17—5.87 3.25-3.79 
11:5 5.87-6.59 3.79—4.35 
12.5 6.59-7.31 4.35—4.93 
13.5 7.31—8.05 4.93-5.52 
14.5 8.05—8.80 5.52-6.12 
15:5 8.80—9.55 6.12-6.74 
16.5 9.55-10.31 6.74—7.37 
17.5 10.31—11.08 7.37-8.01 
18.5 11.08—11.85 8.01-8.66 
19.5 11.85-12.63 8.66-9.31 
20.5 12.63—13.42 9.31-9.98 
21,5 13.42—14.21 9.98-10.65 
22.5 14.21—15.00 10.65-11.33 
23.5 15.00-15.80 11.33-12.02 
24.5 15.80-16.61 12.02-12.71 
25.5 16.61—17.41 12.71-13.41 
26.5 17.41-18.23 13.41-14.11 
27.5 18.23-19.04 14.11-14.82 
28.5 19.04—19.86 14.82-15.53 
29.5 19.86—20.68 15.53-16.25 
30.5 16.25-16.98 
31:5 16.98—17.70 
32.5 17.70-18.44 
33.5 18.44—19.17 
34.5 19.17-19.91 
35.5 19.91—20.66 


An alternative practice (especially popular in the USA) is to dispense with the warn- 
ing limit and to set the action limit (called the upper control limit, UCL) at three 
standard deviations above the mean. Because the count of defectives is binomial with 
mean np and variance np(1 — p), this means that 


UCL = np + 3\ [np — p)] 
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Solution 


113.4 


Example 11.27 


Solution 
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Find the UCL and apply it to the data in Example 11.25. 


From n= 50 and p = 0.05 we infer that UCL = 7.1, which is between the warning limit 
Cw and the action limit c, in Example 11.25. The decision to correct the process would 
be taken after the 15th sample, the first to exceed the UCL. 


Shewhart variable control charts 


Suppose now that the appropriate assessment of quality involves measurement on a 
continuous scale rather than success or failure under some criterion. This arises when- 
ever some dimension of the output is critical for applications. Again we take samples, but 
this time we measure this critical dimension and average the results. The Shewhart chart 
for this variable measure is a plot of successive sample averages against sample number. 

The warning and action limits cy and c, on a Shewhart chart are those points for 
which the probabilities of a false alarm (where the result exceeds the limit even though 
the process is in control) are one in 40 and one in 1000 respectively. For variable measure 
the critical quantity can be either too high or too low, so the sample average must be tested 
in each direction with the stated probability of exceedance for each limit. It follows that 
the limits are determined by 


PX > y+ cy) = PX < py- cw) =% 
PX > yt c,)= PX < by-e) = 


1000 


where X is the sample average and [Uy the design mean. 
Provided that the sample size n is not too small, the central limit theorem allows the 
sample average to be assumed normal (Section 11.3.2), 


A N(LLx, o;/n) 
and the normal distribution table (Figure 11.2) then gives 


_1960y 3.096 
a aan | a 


Cw ; ; 
yn yn 


Measurements of sulphur dioxide concentration (in ug m™*) in the air are taken daily at 
five locations, and successive average readings are as follows: 


64.2, 56.9, 57.7, 67.9, 61.7, 59.7, 55.6, 63.7 
58.3, 66.4, 67.2, 65.2, 63.1, 67.6, 64.1, 66.7 


It is suspected that the mean increased during that time. Assuming normal data with a 
long-term mean of 60.0 and standard deviation of 8.0, investigate whether an increase 
occurred. 


From n= 5 and oy = 8 we have cy = 7.0 and c, = 11.1 (Figure 11.30). The warning limit 
is 67.0, which is exceeded by sample numbers 4, 11 and 14. The action limit is 71.1, 
which is not exceeded. The readings are suspiciously high — but not sufficiently so for 
the conclusion to be justified. 
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Figure 11.30 
Variable control chart 
for Example 11.27. 


11.9.4 


Concentration/ug m> § 


Action limit 


Warning limit 


64+ 


60 


Sample number 


56 +7 


52 + Warning limit 


48 + Action limit 


As discussed in Section 11.9.2, the practice in the USA is somewhat different: there 
are no warning limits, only action limits at three standard deviations on either side of 
the mean. For a variable chart this allows a deviation from the mean of at most 30y/\n, 
which is very close to the action limit usually used in the UK. 


Cusum control charts 


The main concern in designing a control chart is to achieve the best compromise between 
speedy detection of a fault on the one hand and avoidance of a proliferation of false alarms 
on the other. If the chart is too sensitive, it will lead to a large number of unnecessary 
shutdowns. The Shewhart charts, on the other hand, are rather conservative in that they 
are slow to indicate a slight but genuine shift in performance away from the design level. 
This derives from the fact that each sample point is judged independently and may well 
lie inside the action limits, whereas the cumulative evidence over several samples might 
justify an earlier decision. Rather informal methods involving repeated warnings and trends 
are used, but it is preferable to employ a more powerful control chart. The cumulative 
sum (cusum) chart achieves this, and is easily implemented on a small computer. 

Suppose that we have a sequence {Y,, ¥5,... } of observations, which may be either 
counts of defectives or sample averages. From this a new sequence {Sp, S,,... } 
is obtained by setting 


Si, = max{0,S,,+ Y, —r} (m=1,2,...) 


where r is a constant ‘reference value’. This gives a cumulative sum of values of Y,, — 7, 
which is reset to zero whenever it goes negative. The out-of-control decision is made when 


S, >A 


m 


www.2Ofile.org 


Figure 11.31 Cusum 
attribute chart control 
data. 


Example 11.28 


Solution 
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np 1 h np r h 

0.22 1 1.5 2.35 4 4.5 
0.39 1 2.5 2.60 4 3.5 
0.51 2 1.5 2.95 5 4.5 
0.62 1 4.5 3.24 5 a5 
0.69 1 5.5 3.89 6 2D) 
0.79 2 2.5 4.16 6 6.5 
0.86 3 1S 5:32 7 8.5 
1.05 2 3.5 6.07 8 8.5 
1.21 3 2.5 7.04 9 O35 
1.52 3 3.5 8.01 10 10.5 
1.96 3 5.5 9.00 11 11.5 
2.16 3) 2.5, 10.00 12 12.5 


where h is a constant ‘decision interval’. This will detect an increasing mean; a separ- 
ate but similar procedure can be used to detect a decreasing mean. Values of r and 
h for both attribute and variable types of control can be obtained from tables such as 
those in J. Murdoch, Control Charts (Macmillan, London, 1979), from which the attribute 
table in Figure 11.31 has been extracted. For variable measure (with process design 
mean [ly and standard deviation oy) the following are often used: 

Or 


fa 2a h=5ok 
yn yn 


Regular samples of 50 are taken from a process making electronic components, for which 
an acceptable proportion of defectives is 5%. Successive counts of defectives in each 
sample are as follows: 


9 10 11 12 13 14 15 16 #17 #18 #19 20 
2 6 7 4 5 5 8 6 5 9 7 8 


At what point would the decision be taken to stop and correct the process? 


The acceptable proportion of defectives is p = 0.05 and the regular sample size is 
n= 50. From the table in Figure 11.31, with np = 2.5 the nearest figures for reference 
value and decision interval are r = 4 and h = 5.5. The following shows the cusum S,, 
below each count of defectives Y,,, and the cusum is also plotted in Figure 11.32: 


8 6 5 9 7 8 


Count 5 5 
6 7 11 13 14 19 22 26 


Cusum 


For example, 
S\j,=S,+¥,-r=54+5-4=6 


and because this exceeds h = 5.5, the decision to take action would be made after the 
13th sample. This result can be compared with that of a Shewhart chart applied to the 
same data (Example 11.25), which suggests that action should be taken after 18 samples. 
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Figure 11.32 
Cusum control chart 
for Example 11.28. 


Example 11.29 


Solution 


Figure 11.33 
Cusum control chart 
for Example 11.29. 


Sin 


O 2 4 6 8 10 12 14 16 18 20 
Sample number m 


Construct a cusum chart for the sulphur dioxide data in Example 11.27. 


From ly = 60, 0, = 8 and n= 5 we have 


p=(e HS ho 5 S79 
2/5 5 


V Vv 
The following table shows the sample average X,, and cusum S,, for 1 < m < 16, and 
the cusum is also plotted in Figure 11.33: 


Average 64.2 56.9 57.7 67.9 61.7 59.7 55.6 63.7 
Cusum 2.4 0 0 6.1 6.0 3.9 0 1.9 
Average 58.3 66.4 67.2 65.2 63.1 67.6 64.1 66.7 
Cusum 0 4.6 10.0 13.4 14.7 20.5 22.8 27.7 


} } — 
O 2 4 6 8 10 12 14 16 


Sample number m 
Because S,, = 20.5 exceeds A = 17.9, this chart suggests that the SO, concentration did 


increase during the experiment, a stronger result than that obtained from the Shewhart 
chart in Example 11.27. 
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It can be shown that the cusum method will usually detect an out-of-control con- 
dition (involving a slight process shift) much sooner than the strict Shewhart method, 
but with essentially the same risk of a false alarm. For instance, the cusum method 
leads to a decision after 13 samples in Example 11.28 compared with 18 samples in 
Example 11.25 for the same data. The measure used to compare the two methods is the 
average run length (ARL), which is the mean number of samples required to detect 
an increase in proportion of defectives (or process average) to some specified level. It 
has been shown that the ARL for the Shewhart chart can be up to four times that for the 
cusum chart (J. Murdoch, Control Charts, Macmillan, London, 1979). 


Moving-average control charts 


The cusum chart shows that the way to avoid the relative insensitivity of the Shewhart 
chart is to allow the evidence of a shift in performance to accumulate over several 
samples. There are also moving-average control charts, which are based upon a 
weighted sum of a number of observations. The best of these, which is very similar to 
the cusum chart in operation, is the geometric moving-average (GMA) chart. This 
will be described here for variable measure, but it also works for attribute measure 
(Exercise 56). 

Suppose that the successive sample averages are X,, X,,..., each from a sample of 
size n. Also suppose that the design mean and variance are Ly and oy. Then the GMA 
is the new sequence given by 


So = Ux 
S, = 1X, + (1-81 (m= 1,2,.. .) 
where 0 < r < | is a constant. The statistical properties of this sequence are simpler 


than for the cusum sequence. First, by successively substituting for S,,_;, S,,-.and so on, 
we can express S,, directly in terms of the sample averages: 


m-1 


Sin ev [Ud ~ Fy Apel - (1 ~ r)"Uy 
i=0 


Then, using the summation formula 


m-1 m 
1 


(lx|< 1) 


it is easy to show (Exercise 57) that the mean and variance of S,, are 


Hs, = E(S,,) = Uy 
" o. 
os, = Var(S,) == [1- (1-7) "1 
m 2-r n 
After the first few samples the variance of S,, tends to a constant value: 
2 
2 ( r } Oy 
0; ~|—|]—_ aasm> © 
me 2-r/ n 


If US practice is followed then the upper and lower control limits can be set at 
(Uy + 30; ). If UK practice is followed then, from the approximate normality of the 
sample averages and the fact that sums of normal random variables are also normal, it 
follows that S,, is approximately normal, so the warning and action limits can be set at 


m 
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Example 11.30 


Solution 


Figure 11.34 Moving- 
average control chart 
for Example 11.30: 

(a) r= 0.2; (b) r=0.4. 


(Uy + 1.960; ) and (Ly + 3.090, ) respectively (although the warning limits now have 
less significance). 

It remains to choose a value for r. If we set r = 1, the whole approach reduces to the 
standard Shewhart charts. Small values of 7 (say around 0.2) lead to early recognition 
of small shifts of process mean, but ifr is too small, a large shift may remain undetected 
for some time. 


Construct GMA charts for the sulphur dioxide data in Example 11.27, using r = 0.2 
and r= 0.4. 


The control charts can be seen in Figure 11.34. Clearly the warning and action limits 
converge fairly quickly to constant values, so little is lost by using those values in 
practice. The warning limit is exceeded from sample 11 for both values of r. The action 
limit is exceeded from sample 14 for r = 0.2 (as for the cusum chart in Example 11.29) 
and at sample 16 for r = 0.4. 


Sm 
64 Action 

Warning 
62 
60 

2 4 6 8 10 12 14 16 
Sample number m 

58 

Warning 
56 Action 

(a) 
Sis 
66 Action 
64 Warning 
62 
60 - 
16 
58 Sample number m 
56 Warning 
54 Action 
(b) 
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Range charts 


The sample range is defined as the difference between the largest and smallest values 
in the sample. The range has two functions in quality control where the quality is of 
the variable rather than the attribute type. First, if the data are normal then the range 
(R, say) provides an estimate 6 of the standard deviation o by 


6=Rid 


where d is a constant that depends upon the sample size n as follows: 


2 3 4 5 6 7 8 9 10 11 12 
1.128 1.693 2.059 2.326 2.534 2.704 2.847 2.970 3.078 3.173 3.258 


It is clearly quicker to evaluate this than the sample standard deviation S, and for the 
small samples typically used in quality control the estimate is almost as good. 

The other reason why the range is important is because the quality of production can 
vary in dispersion as well as (or instead of) in mean. Control charts for the range R are 
more commonly used than charts for the sample standard deviation S when monitoring 
variability within the manufacturing process, and all three types of chart discussed 
above (Shewhart, cusum and moving-average) can be applied to the range. Range 
charts (or R charts) are designed using tables that can be found in specialized books 
on quality control, for example D. C. Montgomery, /ntroduction to Statistical Quality 


Control, 2nd edn, Wiley, New York, 1991. 


11.9.7. Exercises 


It is intended that 90% of electronic devices 50 A bottling plant is supposed to fill bottles with 
emerging from a machine should pass a simple 568 ml (one imperial pint) of liquid. The standard 
on-the-spot quality test. The numbers of defectives deviation of the quantity of fill is 3 ml. Regular 
among samples of 50 taken by successive shifts are samples of 10 bottles are taken and their contents 
as follows: measured. After subtracting 568 from the sample 

5,8, 11, 5,6, 4,9,7, 12,9, 10, 14 averages, the results are as follows: 
Find the action and warning limits, and the sample ~0.2, 1.3, 2.1, 0.3, —0.8, 1.7, 1.3, 0.6, 2.5, 
number at which an out-of-control decision is 1.4, 1.6, 3.0 
taken. Also find the UCL (US practice) and the Using a Shewhart control chart, determine whether 
sample number for action. the mean fill requires readjustment. 
Thirty-two successive samples of 100 castings : ; 

51 ‘Average reverse-current readings (in nA) for 


each, taken from a production line, contained the 
following numbers of defectives 


3, 3, 5, 3, 5, 0, 3, 1, 3, 5, 4, 2, 4, 3, 5, 4 

3, 4, 5, 6, 5, 6, 4, 4, 7, 5, 4, 8, 5, 6, 6, 7 

If the proportion that are defective is to be 
maintained at 0.02, use the Shewhart method 
(both UK and US standards) to indicate whether 
this proportion is being maintained, and if not then 
give the number of samples after which action 
should be taken. 


www.2Ofile.org 


samples of 10 transistors taken at half-hour 
intervals are as follows: 


12.8, 11:2, 13.4, 12.1, 13.6, 13.9, 12.3, 12.9, 
13.8, 13.1, 12.9, 14.0, 13.7, 13.4, 14.2, 13.1, 
14.0, 14.0, 15.1, 14.3 


The standard deviation is 3nA. At what point, if 
any, does the Shewhart control method indicate that 
the reverse current has increased from its design 
value of 12nA? 
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52 


53 


54 
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56 


Using the data in Exercise 50, apply (a) a cusum 
control chart and (b) a moving-average control 
chart with r = 0.3. 


Using the data in Exercise 51, apply (a) a cusum 
control chart and (b) a moving-average control 
chart with r = 0.3. 


Apply a cusum control chart to the data in 
Exercise 48. 


Apply a cusum control chart to the data in 
Exercise 49. 


The diameters of the castings in Exercise 49 are 
also important. Twelve of each sample of 100 were 
taken, and their diameters measured and averaged. 
The differences (in mm) between the successive 
averages and the design mean diameter of 125 mm 
were as follows: 


0.1, 0.3, -0.2, 0.4, 0.1, 0.0, 0.2, -0.1, 0.2, 
0.4, 0.5, 0.1, 0.4, 0.6, 0.3, 0.4, 0.3, 0.6, 0.5, 
0.4, 0.2, 0.3, 0.5, 0.7, 0.3, 0.1, 0.6, 0.5, 0.6, 
0.7, 0.4, 0.5 


Use (a) Shewhart, (b) cusum and (c) moving- 
average (with r = 0.2) control methods to test for 
an increase in actual mean diameter, assuming a 
standard deviation of 1 mm. 


7) 


58 


BE) 


Prove that the mean and variance of the geometric 
moving-average S,, defined in Section 11.9.5 for 
variable measure are given by 


E(S,,) = Uy 


2 
Ox 


os = Var(S,,) =——[1-(1-7r)""] 
m 2-r n 


Suppose that the moving-average control chart is to 
be applied to the counts of defectives in attribute 
quality control. Find the mean and variance of S,, 
in terms of the sample size n, the design proportion 
of defectives p and the coefficient r. Following 

US practice, set the upper control limit at three 
standard deviations above the mean, and apply the 
method to the data in Example 11.28, using r = 0.2. 


The design diameter of a moulded plastic 
component is 6.00 cm, with a standard deviation 
of 0.2 cm. The following data consist of successive 
averages of samples of 10 components: 


6.04, 6.12, 5.99, 6.02, 6.04, 6.11, 5.97, 6.06, 
6.05, 6.06, 6.17, 6.03, 6.13, 6.05, 6.17, 5.97, 
6.07, 6.14, 6.03, 5.99, 6.10, 6.01, 5.96, 6.12, 
6.02, 6.20, 6.11, 5.98, 6.02, 6.12 


After how many samples do the Shewhart, cusum 
and moving-average (with r = 0.2) control methods 
indicate that action is needed? 


ini §=Poisson processes and the theory of queues 


Probability theory is often applied to the analysis and simulation of systems, and this 
can be a valuable aid to design and control. This section, which is therefore applied 
probability rather than statistics, will illustrate how this progresses from an initial math- 
ematical model through the analysis stage to a simulation. 


11.10.1 


Typical queueing problems 


Queues are everywhere: in banks and ticket offices, at airports and seaports, traffic 
intersections and hospitals, and in computer and communication networks. Somebody 
has to decide on the level of service facilities. The problem, in essence, is that it is 
costly to keep customers waiting for a long time, but it is also costly to provide enough 
service facilities so that no customer ever has to wait at all. Queues of trucks, aero- 
planes or ships may be costly because of the space they occupy or the lost earnings 
during the idle time. Queues of people may be costly because of lost productivity or 
because people will often go elsewhere in preference to joining a long queue. Queues 
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Figure 11.35 
A typical queueing 
system. 


11.10.2 


Figure 11.36 
Random events (x) 
on a time axis. 
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Arrivals 


Departures 
Service 


channels 


of jobs or packets in computer networks are costly in loss of time-efficiency. Service 
facilities are costly in capital, staffing and maintenance. Probabilistic modelling, often 
combined with simulation, allows performance evaluation for queues and networks, 
which can be of great value in preparing the ground for design decisions. 

The mathematical model of a simple queueing system is based on the situation shown 
in Figure 11.35. Customers join the queue at random times that are independent of 
each other — the inter-arrival time (between successive arrivals) is a random variable. 
When a service channel is free, the next customer to be served is selected from the 
queue in a manner determined by the service discipline. After being served the customer 
departs from the queueing system. The service time for each customer is another 
random variable. The distributions of inter-arrival time and service time are usually 
assumed to take one of a number of standard patterns. The commonest assumption 
about service discipline is that the next customer to be served is the one who has been 
queueing the longest time (first in, first out). 

The queueing system may be regarded from either a static or a dynamic viewpoint. 
Dynamically, the system might start from an initial state of emptiness and build up with 
varying rates of arrivals and varying numbers of service channels depending upon 
queue length. This is hard to deal with mathematically, but can be treated by computer 
simulation. Useful information about queues can, however, be obtained from the static 
viewpoint, in which the rate at which arrivals occur is constant, as is the number of 
service channels, and the system is assumed to have been in operation sufficiently long 
to have reached a steady state. At any time the queue length will be a random variable, 
but the distribution of queue length is then independent of time. 

We need to find the distributions of queue length and of waiting time for the cus- 
tomer, and how these vary with the number of service channels. Costs can be worked 
out from these results. 


Poisson processes 


Consider the arrivals process for a queueing system. We shall assume that the customers 
join the queue at random times that are independent of each other. Other assumptions 
about the pattern of arrivals would give different results, but this is the most common 
one. We can therefore think of the arrivals as a stream of events occurring at random 
along a time axis, as depicted in Figure 11.36. The inter-arrival time T, say, will be 
a continuous random variable with probability density function f,(4) and cumulative 
distribution function F;.(f). 


Time 
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Example 11.31 


Solution 


One way to formulate the assumption of independent random arrivals is to assert that 
at any moment the distribution of the time until the next arrival is independent of the 
time elapsed since the previous arrival (because arrivals are ‘blind’ to each other). This 
is known as the memoryless property, and can be expressed as 


P(T<t+h|T>)=P(T<h) (t,h=0) 


where ¢ denotes the actual time since the previous arrival and h denotes a possible time 
until the next arrival. Using the definition of conditional probability (Section 11.2.1), 
we can write this in terms of the distribution function F;,(f) as 


Pit< Tx pe) Pe 


1-P(T Sh) 1 - F;(t) 
Rearranging at the second equality and then dividing through by / gives 


Fy (h 
HF (t+ h) = Fe()] = Her - F,(0)] 
Letting h — 0, we obtain a first-order linear differential equation for F;(f): 
£ F(t) = ALL - F(0)] 
dt 


where 


iin 


hoo 


With the initial condition F;,(0) = 0 (because inter-event times must be positive), the 
solution is 


F,()=1-e™ (t2 0) 


and hence the probability density function is 
frlt)= LF = he" (42 0) 


This is the density function of an exponential distribution with parameter /, and it 
follows that the mean time between arrivals is 1/A (see Section 11.7.1). The parameter 
A is the rate of arrivals (number per unit time). 


A factory contains 30 machines of a particular type, each of which breaks down every 
100 operating hours on average. It is suspected that the breakdowns are not independ- 
ent. The operating time intervals between 10 consecutive breakdowns (of any machine) 
are measured and the shortest such interval is only six minutes. Does this lend support 
to the suspicion of non-independent breakdowns? 


Collectively, the machines break down at the rate of 30/100 or 0.3 per hour. If the 
breakdowns are independent then the interval between successive breakdowns will 
have an exponential distribution with parameter 0.3. The probability that such an interval 
will exceed six minutes is 
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Solution 
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roy 


P(interval > 0.1) = | 0.3e°"dt=e °°” = 0.9704 


0.1 


and the probability that all nine intervals (between 10 breakdowns) will exceed this 
time is (0.9704) = 0.763. Hence the probability that the shortest interval will be six 
minutes or less is one minus this, or 0.237. This is quite likely to have happened by 
chance, so it does not support the suspicion of non-independent intervals. 


The assumption of independent random arrivals therefore leads to a particular dis- 
tribution of inter-arrival time, parametrized by the rate of arrivals. Two further 
conclusions also emerge. First, the number of arrivals that occur during a fixed interval 
of length H has a Poisson distribution with parameter 1H: 

k —AH 
P(k arrivals during interval of length H) = Se (kK=0,1,2,...) 
This will not be proved here, but is easily seen to be consistent with an exponential 
distribution of inter-arrival time 7 because 


F,(t)=P(T<th=1-P(T> 1 
= 1 — P (no event during interval of length #) 
=l-e 
using the Poisson distribution. Because of this distribution, events conforming to these 
assumptions are known as a Poisson process. 

The other conclusion is that the probability that an arrival occurs during a short 
interval of length h is equal to Ah + O(h’), regardless of the history of the process. 
Suppose that a time ¢ has elapsed since the previous arrival, and consider a short interval 
of length / starting from that point: 

P(arrival during (t,¢+ A))=P(T St+h|T> H=F;,(A) 
=1-e“=Ah+ O(h’) 
Ah 


using the memoryless property and the expansion of e*" to first order. Furthermore, the 
probability of more than one arrival during a short interval of length h is O(h’). 


A computer receives on average 60 batch jobs per day. They arrive at a constant rate 
throughout the day and independently of each other. Find the probability that more than 
four jobs will arrive in any one hour. 


The assumptions for a Poisson process hold, so the number of jobs arriving in one hour 
is a Poisson random variable with parameter 1H = 60/24. Hence 


P(more than four jobs) = 1 — P(0 or | or 2 or 3 or 4 jobs) 


2 3 4 
=l-e% 1+ ans GRD, GED, GAD = 0.109 
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11.10.3 


Single service channel queue 


Consider a queueing system with a Poisson arrival process with mean rate A per unit 
time, and a single service channel. The behaviour of the queueing system depends not 
only on the arrival process but also upon the distribution of service times. A common 
assumption here is that the service time distribution (like that of inter-arrival time) is 
exponential. Thus the probability density function of service time S is 


fs(s)=He™ (s = 0) 


Unlike the inter-arrival time distribution in Section 11.10.2, this is not based on an 
assumption of independence or the memoryless property, but simply on the fact that in 
many queueing situations most customers are served quickly but a few take a lot longer, 
and the form of the distribution conforms with this fact. This assumption is therefore 
on much weaker ground than that for the arrival time distribution. The parameter [ is 
the mean number of customers served in unit time (with no idle periods), and the mean 
service time is 1/1. With this service distribution, the probability that a customer in the 
service channel will have departed after a short time / is equal to uh + O(h’), independent 
of the time already spent in the service channel. 


Distribution of the number of customers in the system 


We can now derive the distribution of the number of customers in the queueing system. 
Considering the system as a whole (queue plus service channel), the number of cus- 
tomers in the system at time f is a random variable. Let p,(t) be the distribution of this 
random variable: 


p,{t) = P(n customers in the system at time tf) (n=0, 1, 2,...) 


Consider the time ¢ + /, where h is small. The probability of more than one arrival or 
more than one departure during this time is O(h’), and will be ignored. There are four 
ways in which there can be n (assumed greater than zero) customers in the system at 
that time: 


(1) there are n in the system at f, and no arrival or departure by ¢ + h; the probability 
of this is given by 


PCL = AR) = wh) + O(h*) = p,(Q( — Ah wh) + OCH") 


(2) there are n in the system at ¢, and one arrival and one departure by ¢ + h; the 
probability is given by 


PyAlt\(Ah)( wh) + O(h*) = Ofh") 


(3) there are n — | in the system at f, and one arrival but no departure by ¢ + A; the 
probability is given by 


Pralt(Ah(l = wh) + Oh") = p, (AA) + O(K’) 


(4) there are n + 1 in the system at f, and no arrivals but one departure by f + A; the 
probability is given by 


Pru — Ah)(Uh) + O17) = Pyi(O(Mh) + O(h") 
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Summing the probabilities of these mutually exclusive events gives the probability of 
n customers in the system at time f+ h as 


+ Pui(D(Uh) + Oh?) (n=1,2,...) (11.1) 
Similarly, there are two ways in which the system can be empty (” = 0) at time ¢ + A: 
empty at ¢ and no arrival before ¢ + h, or one customer at ¢ who departs before ¢ + h. 
This gives 
Polt + h) = pot — Ah) + p()( Uh) + O(n’) (11.2) 
Rearranging equations (11.1) and (11.2) and taking the limit as A > 0, we obtain 


£ pald) = tim ¢Lea(t+h) ~ p,(0)] 


hoo 


=-(A+ U)p,(t) a Ap, (t) + UP i(t) (n — 1, 2s aieee ) 
£ po(t) = ~Apo(t) + Upi(t) 


This is a rather complex set of recursive differential equations for the probabilities p,(f). 
If we assume that the arrival and service parameters A and wu are constant and that the 
system has been in operation for a long time then the distribution will not depend upon ¢; 
the derivatives therefore vanish, and we are left with the following algebraic equations 
for the steady-state distribution p,,: 

O=—-(A+ Bp, + Apr + UP ost (n=1,2,...) 

0=—Apy + MP; 
Defining the ratio of arrival and service parameters A and yu as p = A/u and dividing 
through by u, we have 

Posi = (1 + P)Pa- PPn-1 (n= 1, 2, oe .) 

Pi =PPo 


To solve these, we first assume that p, = "py. Clearly this works for n = 0 andn = 1. 
Substituting, 


n+] 


Poi = (1 + p)p"Po— Pp” 'Po = p”™'Po 


so the assumed form holds for n+ 1, and therefore for all m by induction. It remains only 
to identify p, from the fact that the distribution must sum to unity overn=0, 1,2,...: 


n Po 
l=p > p == 
Hence py = | — p and 
P,=(1-p)p' (n=0,1,2,...) 


This is known as the geometric distribution, and is a discrete version of the exponen- 
tial distribution (Figure 11.37). Note that this result requires that p < 1, or equivalently 
A < uw. If this condition fails to hold, the arrival rate swamps the capacity of the service 
channel, the queue gets longer and longer, and no steady-state condition exists. 
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Figure 11.37 
Geometric distribution 
(with p = 0.75). 


0.25 


Queue length and waiting time 
The queue length distribution now follows easily: 
P(queue empty) = pot P; 
=l1-9° 
P(n in queue) = P(n + | in system) 
=(l=pjp" @=1,2....) 


Denoting the mean numbers of customers in the system and in the queue by Ng and No 
respectively, 


2 


No= Yi mpa= TEs, No= Y1- Den = To 
n=0 n=1 


(Exercise 62). Since in the steady state the mean time between departures must equal 
the mean time between arrivals (1/A), it is plausible that the mean total time in the 
system for each customer, Wg say, is given by 


W,; = mean number in system X mean time between departures 
_plA __1 
l-p w-aA 
The mean waiting time in the queue, Wg say, is then 


W. = mean time in system — mean service time 


= W,- 1__P 
Hh p-A 
These results for W; and Wg can be derived more formally from the respective waiting 
time distributions. For example, the distribution of total time in the system can be 
shown to be exponential with parameter  — A, and the waiting time in the queue can 
be expressed as 


P(waiting time in queue = t))=pe“” (t> 0) 


www.2Ofile.org 


Example 11.33 


Solution 


Example 11.34 


Solution 
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If customers in a shop arrive at a single check-out point at the rate of 30 per hour and if 
the service times have an exponential distribution, what mean service time will ensure 
that 80% of customers do not have to wait more than five minutes in the queue and 
what will be the mean queue length? 


With A = 0.5 and t= 5, the queue waiting time gives 
0.2= pe 

that is, 
0.2u = 0.5 e° > *# 


This is a nonlinear equation for 4, which may be solved by standard methods to give 
Lt = 0.743. The mean service time is therefore 1/u or 1.35 min, and the mean queue 
length is 


No=— = 139, using p = 7 = 0.673 


Handling equipment is to be installed at an unloading bay in a factory. An average of 
20 trucks arrive during each 10h working day, and these must be unloaded. The 
following three schemes are being considered: 


Scheme Fixed cost/ Operating cost/ Mean handling rate/ 
£ per day £ per hour trucks per hour 

A 90 45 3 

B 190 50 4 

C 450 60 6 


Truck waiting time is costed at £30 per hour. Assuming an exponential distribution of 
truck unloading time, find the best scheme. 


Viewing this as a queueing problem, we have 

A = arrival rate per hour = 2.0 

= unloading rate per hour 

mean waiting time for each truck = 1/(u — A) 
Hence the mean delay cost per truck is 


30 
u-2 
and the mean delay cost per day is 
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11.10.4 


Example 11.35 


The proportion of time that the equipment is running is equal to the probability that the 
system is not empty (the utilization), which is 


A 
l-p —a 
ae 


Hence the mean operating cost per day is 10p times operating cost per hour. The total 
cost per day (in £) is the sum of the fixed, operating and delay costs, as follows: 


Scheme u p Fixed Operating Delay Total 
A cS) 0.6667 90 300 600 990 
B 4 0.5 190 250 300 740 
Cc 6 0.3333 450 200 150 800 


Hence scheme B minimizes the total cost. 


Queues with multiple service channels 


For the case where there are c service channels, all with an exponential service time 
distribution with parameter j, a line of argument similar to that in Section 11.10.3 can 
be found in many textbooks on queueing theory. In particular, it can be shown that the 
distribution p, of the number of customers in the system is 


-1 


“ ni (= TNe= p) 


n=' 


The mean numbers in the queue and in the system are 


c+1 


p 
No =——+———- p>, Ns=Not+p 
“(e=Ie-py 
and the mean waiting times in the queue and in the system are 
N 1 
Wo= rie W,= Uae 


For the unloading bay problem in Example 11.34 a fourth option would be to install 
two sets of equipment under scheme A (there is space available to do this). The fixed 
costs would then double but the operating costs per bay would be the same. Evaluate 
this possibility. 
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With two bays under scheme A, we have A = 2, u = 3 and c = 2, so that p = 2, and the 
probability that the system is empty at any time is 


-1 
Pp 1 
po=(1+p+ } =; 
0 p 2 


2. = 


The probabilities of one truck (one bay occupied) and of two or more trucks (both bays 
occupied) are then 


P,= Ppo= 3 


P(two or more trucks) = | - 


tla 
23° 6 


The total operating cost per day is the operating cost for when one or other bay is working 
(£45 per hour) plus that for when both bays are working (£90 per hour), which is 


10[4 (45) + £(90)] = 300 


The mean number in the queue is 
3 


p 
(2-p) 
so that the mean total time in the system for each truck is 


5 (0.083 33) + + = 0.375 


Py = 0.083 33 


2 


Multiplying by the cost per hour and the number of trucks gives the delay cost per day: 
20(30)(0.375) = 225 

The total cost per day of this scheme is therefore 
2(90) + 300 + 225 = £705 


This is less than the £740 under scheme B, the best of the single-bay options. 


Queueing system simulation 


The assumption that the service time distribution is exponential, which underlies the 
results in Sections 11.10.3 and 11.10.4, is often unrealistic. It is known that it leads to 
predicted waiting times that tend to be pessimistic, as a result of which costs based 
on these predictions are often overestimated. Theoretical results for other service dis- 
tributions exist (see for example E. Page, Queueing Theory in OR. Butterworth, London, 
1972), but it is often instructive to simulate a queueing system and find the various 
answers numerically. It is then easy to vary the arrival and service distributions, and the 
transient (non-steady-state) behaviour of the system also reveals itself. 

Figure 11.38 shows a pseudocode listing of a single-channel queueing system simula- 
tion, which is easily modified to cope with multiple channels. Each event consists of either 
an arrival or a departure. The variables next_arrival and next_departure are used to 
represent the time to the next arrival and the time to the next departure respectively, 
and the type of the next event is determined by whichever is the smaller. New arrival 
and departure times are returned by the functions arrival_time and departure_time, 
which generate values from appropriate exponential distributions. The distribution of 
the number of customers in the system is built up as an array, normalized at the end of 
the simulation, from which the mean and other results can be obtained. 


www.2Ofile.org 


984 APPLIED PROBABILITY AND STATISTICS 


Figure 11.38 
Pseudocode listing 
for queueing system 
simulation. 


{ Procedure to simulate a single-channel queueing system. 

time is the running time elapsed, 

limit is the simulation length, 

number is the number of items in the system, 

maximum is the maximum number allowed in the system, 

system[i] is the distribution (array) of times with i in the 
system, assumed initialized to zero, 

mean is the mean number in the system, 

infinity contains a very large number, 

arrival_rate is the arrival distribution parameter, 

service_rate is the service distribution parameter, 

next_arrival is the time to the next arrival, 

next_departure is the time to the next departure, 

rnd() is a function assumed to return a uniform random (0,1) 
number. 

The simulation starts with the system empty, 

limit, maximum, arrival_rate and service_rate must be set. } 


time — 0 
next_arrival < arrival_time(arrival_rate) 
next_departure < infinity 
number <— 0 
repeat 
if next_arrival < next_departure then 
time < time + next_arrival 


system[number] < system[number] + next_arrival 
arrival() 


else 
time < time + next_departure 
system[number] < system[number] + next_departure 
departure() 
endif 
until time > limit 
mean < 0 
for iis 0 to maximum do 
system[i] <— system[i]/time 
mean < mean + i* system[i] 
endfor 
a 2 aE aC a a oa ae a a a ae 


procedure arrival() 
{procedure to handle an arrival, 
changes the values of next_arrival, next_departure, and number 
if number = 0 then 
next_departure < departure_time(service_rate) 
else 
next_departure <— next_departure — next_arrival 
endif 
number < number + | 
if number = maximum then 
next_arrival < infinity 
else 
next_arrival < arrival_time(arrival_rate) 
endif 
endprocedure 
SCR a 2 2 a a a a ak 2 ck a ak ae 
procedure departure() 
{ procedure to handle a departure, 
changes the values of next_arrival, next_departure, and number 
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if number = maximum then 
next_arrival < arrival_time(arrival_rate) 
else 
next_arrival <— next_arrival — next_departure 
endif 
number <— number — | 
if number = 0 then 
next_departure < infinity 
else 
next_departure < departure_time(service_rate) 
endif 
endprocedure 
oie 2 2 2 2 2 2 2 2 2 ok oo ok KK KK 
function arrival_time(arrival_rate) 
{ function to generate a new arrival time } 
U< md) 
return( — (log(U))/arrival_rate) 
endfunction 
function departure_time(service_rate) 
{ function to generate a new departure time } 
U< md) 
return ( — (log/U))/service_rate) 
endfunction 


What is typically found from such a simulation (with Poisson arrivals and an exponen- 
tial service distribution) is that there is good agreement with the predicted results as long 
as p = A/u is small, but the results become more erratic as p > |. It takes a very long 
time for the distribution to reach its steady-state form when the value of p is close to 
unity. In that situation the theoretical steady-state results may be of limited value. 


60 


61 
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11.10.6 Exercises 


A sea area has on average 15 gales annually, evenly 
distributed throughout the year. Assuming 
that the gales occur independently, find the 
probability that more than two gales will occur 

in any one month. 


Suppose that the average number of telephone calls 
arriving at a switchboard is 30 per hour, and that 
they arrive independently. What is the probability 
that no calls will arrive in a three-minute period? 
What is the probability that more than five calls will 
arrive in a five-minute period? 


Show that for a single-channel queue with Poisson 
arrivals and exponential service time distribution 
the mean numbers of customers in the system and in 
the queue are 


= ti 
Mao Mats 
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where p is the ratio of arrival and service rates. 
(Hint: Differentiate the equation 


with respect to p.) 


Patients arrive at the casualty department of a 
hospital at random, with a mean arrival rate of 
three per hour. The department is served by one 
doctor, who spends on average 15 minutes with 
each patient, actual consulting times being 
exponentially distributed. Find 


(a) the proportion of time that the doctor is idle; 
(b) the mean number of patients waiting to 
see the doctor; 
(c) the probability of there being more than three 
patients waiting; 
(d) the mean waiting time for patients; 
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(e) the probability of a patient having to wait 
longer than one hour. 


A small company operates a cleaning and 
re-catering service for passenger aircraft at an 
international airport. Aircraft arrive requiring this 
service at a mean rate of A per hour, and arrive 
independently of each other. They are serviced one 
at a time, with an exponential distribution of service 
time. The cost for each aircraft on the ground is put 
at c, per hour, and the cost of servicing the planes at 
a rate [is c)U per hour. Prove that the service rate 
that minimizes the total cost per hour is 


The machines in a factory break down in a Poisson 
pattern at an average rate of three per hour during 
the eight-hour working day. The company has two 
service options, each involving an exponential 
service time distribution. Option A would cost 
£20 per hour, and the mean repair time would 


66 


67 


be 15 min. Option B would cost £40 per hour, with 
a mean repair time of 12 min. If machine idle time 
is costed at £60 per hour, which option should be 
adopted? 


Ships arrive independently at a port at a mean 

rate of one every three hours. The time a ship 
occupies a berth for unloading and loading has an 
exponential distribution with a mean of 12 hours. 
If the mean delay to ships waiting for berths is to be 
kept below six hours, how many berths should there 
be at the port? 


In a self-service store the arrival process is Poisson, 
with on average one customer arriving every 30s. 
A single cashier can serve customers every 48 s on 
average, with an exponential distribution of service 
time. The store managers wish to minimize the 
mean waiting time for customers. To do this, they 
can either double the service rate by providing an 
additional server to pack the customer’s goods 
(at a single cash desk) or else provide a second cash 
desk. Which option is preferable? 


iui Bayes’ theorem and its applications 


To end this chapter, we return to the foundations of probability and inference. The 
definition of conditional probability is fundamental to the subject, and from it there 
follows the theorem of Bayes, which has far-reaching implications. 


11.11.1 Derivation and simple examples 


The definition in Section 11.2.1 of the conditional probability of an event B given that 
another event A occurs can be rewritten as 


P(A NM B)= P(B|A)P(A) 
If A and B are interchanged then this becomes 

P(A NM B)= P(A|B)P(B) 
The left-hand sides are equal, so we can equate the right-hand sides and rearrange, 
giving 

P(A|B) = aa 4 


Now suppose that B is known to have occurred, and that this can only happen if one of 
the mutually exclusive events 


{A,,...,4,}, 4,04, =O (i#/) 
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Example 11.36 


Solution 


11.11 BAYES’ THEOREM AND ITS APPLICATIONS 987 


has also occurred, but which one is not known. The relevance of the various events 4; 
to the occurrence of B is expressed by the conditional probabilities P(B|A;). Suppose 
that the probabilities P(A;) are also known. The examples below will show that this is 
a common situation, and we should like to work out the conditional probabilities 


P(B|A,)P(A)) 
P(B) 


To find the denominator, we sum from | to n: 


P(A,|B) = 


¥ PUB) =l= ¥ PBA) PCA) 


a 
P(B) 
The sum is equal to | by virtue of the assumption that B could not have occurred without 
one of the A; occurring. We therefore obtain a formula for P(B): 


P(B) = PCBIA)PCA) 


i=] 


which is sometimes called the rule of total probability. Hence we have the following 
theorem. 


Bayes’ theorem 


If {A,,..., A,} are mutually exclusive events, one of which must occur given that 
another event B occurs, then 
P(A,|B) = oo eee (see) 
>) P(BIA/) P(A) 


jal 


end of theorem 


Three machines produce similar car parts. Machine A produces 40% of the total output, 
and machines B and C produce 25% and 35% respectively. The proportions of the 
output from each machine that do not conform to the specification are 10% for A, 5% 
for B and 1% for C. What proportion of those parts that do not conform to the 
specification are produced by machine A? 


Let D represent the event that a particular part is defective. Then, by the rule of total 
probability, the overall proportion of defective parts is 


P(D) =P(D|A)P(A) + P(D|B)P(B) + P(D|C)P(C) 
= (0.1)(0.4) + (0.05)(0.25) + (0.01)(0.35) = 0.056 


Using Bayes’ theorem, 


P(A|D) = L2 A)P(A) _ (0.1)(0.4) _ 9 744 


P(D) 0.056 
so that machine A produces 71.4% of the defective parts. 


www.2Ofile.org 


988 APPLIED PROBABILITY AND STATISTICS 


Example 11.37 


Solution 


LiL 


Suppose that 0.1% of the people in a certain area have a disease D and that a mass 
screening test is used to detect cases. The test gives either a positive or a negative result 
for each person. Ideally, the test would always give a positive result for a person who 
has D, and would never do so for a person who has not. In practice the test gives a 
positive result with probability 99.9% for a person who has D, and with probability 
0.2% for a person who has not. What is the probability that a person for whom the test 
is positive actually has the disease? 


Let 7 represent the event that the test gives a positive result. Then the proportion of 
positives is 


P(T) = P(T|D)P(D) + P(T|D)P(D) 
= (0.999)(0.001) + (0.002)(0.999) = 0.003 


and the desired result is 


P(D|T) = PLIPYPC) _ (0.999)(0.001) _ 1 
P(T) 0.003 : 


Despite the high basic reliability of the test, only one-third of those people receiving a 
positive result actually have the disease. This is because of the low incidence of the 
disease in the population, which means that a positive result is twice as likely to be a 
false alarm as it is to be correct. 


In connection with Example 11.37, it might be wondered why the reliability of the 
test was quoted in the problem in terms of 


P(positive result|disease) and P(positive result|no disease) 
instead of the seemingly more useful 
P(disease | positive result) and P(disease | negative result) 


The reason is that the latter figures are contaminated, in a sense, by the incidence of the 
disease in the population. The figures quoted for reliability are intrinsic to the test, and 
may be used anywhere the disease occurs, regardless of the level of incidence. 


Applications in probabilistic inference 


The scope for applications of Bayes’ theorem can be widened considerably if we assume 
that the calculus of probability can be applied not just to events as subsets of a sample 
space but also to more general statements about the world. Events are essentially state- 
ments about facts that may be true on some occasions and false on others. Scientific 
theories and hypotheses are much deeper statements, which have great explanatory 
and predictive power, and which are not so much true or false as gaining or lacking in 
evidence. One way to assess the extent to which some evidence E supports a hypothesis 
/7is in terms of the conditional probability P(H| £). The relative frequency interpretation 
of probability does not normally apply in this situation, so a subjective interpretation is 
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Example 11.38 


Solution 


Example 11.39 
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adopted. The quantity P(H| F) is regarded as a degree of belief in hypothesis H on the 
basis of evidence £. In an attempt to render the theory as objective as possible, the rules 
of probability are strictly applied, and an inference mechanism based on Bayes’ theorem 
is employed. 

Suppose that there are in fact two competing hypotheses H, and H,. Let X represent 
all background information and evidence relevant to the two hypotheses. The probabil- 
ities P(H, |X) and P(A, |X M £) are called the prior and posterior probabilities of H,, 
where E is anew piece of evidence. Similarly, there are prior and posterior probabilities 
of H,. Applying Bayes’ theorem to both H, and H, and cancelling the common denomin- 
ator P(E) gives 


P(A|XNE) _ P(E|H, OX) PUA, |X) 
P(H,|XNE) P(E|H, 1X) P(H,|X) 


The left-hand side and the second factor on the right-hand side are called the posterior 
odds and prior odds respectively, favouring H, over H,. The first factor on the right- 
hand side is called the likelihood ratio, and it measures how much more likely it is that 
the evidence event E would occur if the hypothesis H, were true than if H, were true. 
The new evidence F therefore ‘updates’ the odds, and the process can be repeated as 
often as desired, provided that the likelihood ratios can be calculated. 


From experience it is known that when a particular type of single-board microcomputer 
fails, this is twice as likely to be caused by a short on the serial interface (H,) as by a 
faulty memory circuit (H,). The standard diagnostic test is to measure the voltage at a 
certain point on the board, and from experience it is also known that a drop in voltage 
there occurs nine times out of ten when the memory circuit is faulty but only once in 
six occasions of an interface short. How does the observed drop in voltage (£) affect 
the assessment of the cause of failure? 


The prior odds are two to one in favour of H,, and the likelihood ratio is (1/6)/(9/10), 
so the posterior odds are given by 


PUA LE) 


PCEE|E) 7 (2) = 0.370 


The evidence turns the odds around to about 2.7 to one in favour of H). 


An oil company is prospecting for oil in a certain area, and is conducting a series of 
seismic experiments. It is known from past experience that if oil is present in the rock 
strata below then there is on average one chance in three that a characteristic pattern 
will appear on the trace recorded by the seismic detector after a test. If oil is absent 
then the pattern can still appear, but is less likely, appearing only once in four tests on 
average. After 150 tests in the area the pattern has been seen on 48 occasions. Assuming 
prior odds of 3:1 against the presence of oil, find the updated odds. Also find the 90% 
confidence interval for the true probability of the pattern appearing after a test, and 
hence consider whether oil is present or not. 
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Solution 


Let H, and H, represent the hypotheses that oil is present and that it is absent respect- 
ively. There were effectively 150 pieces of evidence gathered, and the odds need to be 
multiplied by the likelihood ratio for each. Each time the pattern is present the likeli- 
hood ratio is 


P(pattern|H)) _ 1/1 _ 4 
P( pattern | H, 


) ~ 3/47 3 
and each time it is absent the likelihood ratio is 


P(no pattern #4) _ 2/3 _ 
P(no pattern | H,) —s 


The updated odds, letting E represent the total evidence, become 


P(A, |E) — 14,48/9,102/ 
—— — = (4) (8) (4) = 2.01 
PULIE) GG) G 
The odds that there is oil present are therefore raised to 2:1 in favour. 

Confidence intervals for proportions were covered in Section 11.3.6. The proportion 
of tests for which the pattern was observed is 48/150 or 0.32, so the 90% confidence 
interval for the probability of appearance is 


0.32+ 1.645, ] ese tOes = (0.26, 0.38) 


The hypothesis that oil is absent is not compatible with this, because the pattern should 
then appear with probability 0.25, whereas the hypothesis that oil is present is fully 
compatible. 


For the problem in Example 11.38 it is conceivable that there could be enough 
repetitions for the relative frequency interpretation to be placed on the probabilities of 
the two hypotheses. In contrast, in Example 11.39 the probability of the presence or 
absence of oil is not well suited to a frequency interpretation, but the subjective inter- 
pretation is available. 

Example 11.39 also provides a contrast between the ‘Bayesian’ and ‘classical’ inference 
approaches. The classical confidence interval appears to lead to a definite result: H, 
is true and H, is false. This definiteness is misleading, because it is possible (although 
not likely) that the opposite is the case, but the evidence supports one hypothesis more 
than the other. The Bayesian approach has the merit of indicating this relative support 
quantitatively. 

One area where Bayesian inference is very important is in decision support and 
expert systems. In classical decision theory Bayesian inference is used to update the 
probabilities of various possible outcomes of a decision, as further information becomes 
available. This allows an entire programme of decisions and their consequences to be 
planned (see D. V. Lindley, Making Decisions, 2nd edn. Wiley, London, 1985). Expert 
systems often involve a process of reasoning from evidence to hypothesis with a Bayesian 
treatment of uncertainty (see for example R. Forsyth, ed., Expert Systems, Principles 
and Case Studies. Chapman & Hall, London, 1984). 
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11.11.3 Exercises 


A telephone-based automated customer care system 
has three main menu options: 45% of customers 
choose option 1, 32% choose option 2, and 23% 
choose option 3. Of those who choose option 1, 
28% eventually get routed to a service agent, as do 
41% of those who choose option 2 and 16% of 
those who choose option 3. What is the overall 
proportion of customers who eventually get 
routed to a service agent? 


An explosion at a construction site could have 
occurred as a result of (a) static electricity, 

(b) malfunctioning of equipment, (c) carelessness 
or (d) sabotage. It is estimated that such an explosion 
would occur with probability 0.25 as a result of (a), 
0.20 as a result of (b), 0.40 as a result of (c) and 0.75 
as a result of (d). It is also judged that the prior 
probabilities of the four causes of the explosion 
are (a) 0.20, (b) 0.40, (c) 0.25, (d) 0.15. Find the 
posterior probabilities and hence the most likely 
cause of the explosion. 


Three marksmen (A, B and C) fire at a target. Their 
success rates at hitting the target are 60% for A, 
50% for B and 40% for C. If each marksman fires 
one shot at the target and two bullets hit it, then 
which is more probable: that C hit the target, or did 
not? 


An accident has occurred on a busy highway 
between city A, of 100 000 people, and city B, of 
200 000 people. It is known only that the victim 

is from one of the two cities and that his name is 
Smith. A check of the records reveals that 10% of 
city A’s population is named Smith and 5% of city 
B’s population has that name. The police want to 
know where to start looking for relatives of the 
victim. What is the probability that the victim is 
from city A? 


In a certain community, 8% of all adults over 50 
have diabetes. Ifa health service in this community 
correctly diagnoses 95% of all persons with 
diabetes as having the disease, and incorrectly 
diagnoses 2% of all persons without diabetes as 
having the disease, find the probabilities that 


(a) the community health service will diagnose an 
adult over 50 as having diabetes, 

(b) a person over 50 diagnosed by the health service 
as having diabetes actually has the disease. 
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A stockbroker correctly identifies a stock as being a 
good one 60% of the time and correctly identifies a 
stock as being a bad one 80% of the time. A stock 
has a 50% chance of being good. Find the 
probability that a stock is good if 


(a) the stockbroker identifies it as good, 
(b) & out of n stockbrokers of equal ability 
independently identify it as good. 


On a communications channel, one of three 
sequences of letters can be transmitted: AAAA, 
BBBB and CCCC, where the prior probabilities 
of the sequences are 0.3, 0.4 and 0.3 respectively. 
It is known from the noise in the channel that the 
probability of correct reception of a transmitted 
letter is 0.6, and the probability of incorrect 
reception of the other two letters is 0.2 for 

each. It is assumed that the letters are distorted 
independently of each other. Find the most 
probable transmitted sequence if ABCA is 
received. 


The number of accidents per day occurring at 

a road junction was recorded over a period of 
100 days. There were no accidents on 84 days, 
one accident on 12 days, and two accidents on 
four days. One hypothesis is that the number of 
accidents per day has a Poisson distribution with 
parameter A (unspecified), and another is that the 
distribution is binomial with parameters n = 3 
and p (unspecified). Use the average number of 
accidents per day to identify the unspecified 
parameters and compare the hypotheses assuming 
that the binomial is initially thought to be twice as 
likely as the Poisson. 


The following multinomial distribution is a 
generalization of the binomial distribution. Suppose 
that there are k distinct possible outcomes of an 
experiment, with probabilities p,,..., p,, and that 
the experiment is repeated 7 times. The probability 
of obtaining a number n, of occurrences of the first 
possible outcome, n, of the second, and so on up to 
n, of the kth is 


n 


! ny 
PUG; 03 (Pi) (Pa) 


.5 1) = 7. 


Suppose now that there are two competing 
hypotheses H, and H,. H, asserts that the 
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probabilities are p,,..., p, as above, and H, 
asserts that they are q,,..., q,. Prove that the 
logarithm of the likelihood ratio is 


PRs 0c MAM) |e (2) 
hb | = n; in| = 
ee ae = qi 


i=l : 


77 ~ According to the design specification, of the 
components produced by a machine, 92% should 
have no defect, 5% should have defect A alone, 
2% should have defect B alone and 1% should 
have both defects. Call this hypothesis H,. The 
user suspects that the machine is producing more 
components (say a proportion p,) with defect B 
alone, and also more components (say a proportion 
Paz) With both defects, but is satisfied that 5% have 
defect A alone. Call this hypothesis H,. Of a sample 
of 1000 components, 912 had no defects, 45 had 
A alone, 27 had B alone and 16 had both. Using 
the multinomial distribution (as in Exercise 76), 
maximize In P(912, 45, 27, 16|H,) with respect 


to p, and p,,, and find the posterior odds assuming 
prior odds of 5:1 in favour of H,. 


78  Itis suggested that higher-priced cars are assembled 
with greater care than lower-priced cars. To 
investigate this, a large luxury model A and a 
compact hatchback B were compared for defects 
when they arrived at the dealer’s showroom. All 
cars were manufactured by the same company. The 
numbers of defects for several of each model were 
recorded: 


A: (5, 4, 3,5, 3, 4} 
B: {8, 6, 8, 9, 5} 


The number of defects in each car can be 

assumed to be governed by a Poisson distribution 
with parameter A. Compare the hypothesis H, that 
A, #A, with H, that ,=A, =A, using the average 
numbers of defects to identify the 2 values and 
assuming no initial preference between the 
hypotheses. 


11.12 Review exercises (1-10) 


1 Eight cases each of 12 bottles of wine from a 
vineyard were tested for evidence of oxidation 
in the wine. Five of the cases were bottled using 
standard corks and, of these, six bottles were 
found to have oxidized. The remaining cases 
were bottled using plastic bungs and, of these, 
three bottles were found to have oxidized. Test 
the hypothesis that there is no difference in the 
proportion of bottles oxidized for the different 
types of cork. 


2 The amplitude d of vibration of a damped pendulum 
is expected to diminish by 
Chee 


Successive amplitudes are measured from a trace as 
follows: 


1.01 2.04 3.12 4.09 5.22 630 7.35 8.39 9.44 10.50 
d|246 1.75 1.26 0.94 0.90 0.79 0.52 0.49 0.31 0.21 


Find a 95% confidence interval for the damping 
coefficient A. 


3 Successive masses of 1 kg were hung from a 
wire, and the position of a mark at its lower 
end was measured as follows: 


Load/kg 0 1 2 3 4 5 6 7 
Position/em | 6.12 6.20 6.26 6.32 6.37 6.44 6.50 6.57 


It is expected that the extension Y is related to the 
force X by 


Y=LX/EA 


where L = 101.4 cm is the length, 

A= 1.62 x 10° cm’ is the area and £ is the 
Young’s modulus of the material. Find a 95% 
confidence interval for the Young’s modulus. 


4 The table in Figure 11.39 gives the intervals, in 
hours, between arrivals of cargo ships at a port 
during a period of six weeks. It is helpful to the 
port authorities to know whether the times of 
arrival are random or whether they show any 
regularity. Fit an exponential distribution to the 
data and test for goodness-of-fit. 
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Figure 11.39 Time interval data for Review exercise 4. 
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When large amounts of data are processed, there 
is a danger of transcription errors occurring (for 
example, a decimal point in the wrong place), which 
could bias the results. One way to avoid this is to 
test for outliers in the data. Suppose that X,..., 
, are independent exponential random variables, 
each with a common parameter A. Let the random 


16.3 
6.5 
5.8 

10.7 

7s 
6.8 
8.8 


variable Y be the largest of these divided by the sum: 


Y= Xnox/ YX: 


It can be shown (V. Barnett and T. Lewis, Outliers 
in Statistical Data. Wiley, Chichester, 1978) that 
the distribution function of Y is given by 


fl/y] 


Fy(y) = roa - ky)" 
k=0 


(i<y<1) 
n 


where [1/y] denotes the integer part of 1/y. For the 
data in the Review exercise 4 (Figure 11.39) test 
the largest value to see whether it is reasonable 

to expect such a value if the data truly have an 
exponential distribution. Find 95% confidence 
intervals for the mean inter-arrival time with this 
value respectively included and excluded from 

the data. 


Language courses in French, German and Spanish 
are offered by an adult learning institute. At the end 
of each course, the students are asked to grade 
their response to the course as either very satisfied, 
fairly satisfied, neutral, fairly dissatisfied, or very 
dissatisfied. After gathering data for several terms 
the results are as follows: 


Grade French German Spanish 
Very satisfied 16 6 ae) 
Fairly satisfied 63 13 76 
Neutral 40 27 60 
Fairly dissatisfied 10 13 32 
Very dissatisfied 3 5 i 


10.7 
6.8 
7.6 
8.3 
el 
shal 
4.8 
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SS) 125 1.6 3.0 @)9) DD) 
42.5 A) 3 spl 2.6 1.0 

6.4 11.3 51.6 15.6 2.6 7.6 
US)s1l 3.6 6.0 0.1 3.1 12) 
10.1 18.8 3.4 0.2 4.9 122) 
13.6 15 y50/ 0.7 Jae 18.8 29.8 
0.3 4.6 49 6.1 33.0 6.5 


Is there evidence of different levels of satisfaction 
with the different courses? 


A surgeon has to decide whether or not to 
perform an operation on a patient suspected of 
suffering from a rare disease. If the patient has 
the disease, he has a 50:50 chance of recovering 
after the operation but only a one in 20 chance of 
survival if the operation is not performed. On the 
other hand, there is a one in five chance that a 
patient who has not got the disease would die as 
a result of the operation. How will the decision 
depend upon the surgeon’s assessment of the 
probability p that the patient has the disease? 
(Hint: Use P(B|A) = P(B|A NM C)P(C) + 
P(B|AM C)P(C), where A and C are 
independent.) 


A factory contains 200 machines, each of which 
becomes misaligned on average every 200h of 
operation, the misalignments occurring at random 
and independently of each other and of other 
machines. To detect the misalignments, a quality 
control chart will be followed for each machine, 
based on one sample of output per machine per 
hour. Two options have been worked out: option 
A would cost £1 per hour per machine, whereas 
option B would cost £1.50 per hour per machine. 
The control charts differ in their average run 
lengths (ARLs) to a signal of action required. 
Option A (Shewhart) has an ARL of 20 for a 
misaligned machine, but will also generate false 
alarms with an ARL of 1000 for a well-adjusted 
machine. Option B (cusum) has an ARL of four 
for a misaligned machine and an ARL of 750 for 
a well-adjusted machine. 

When a control chart signals action required, 
the machine will be shut down and will join a 
queue of machines awaiting servicing. A single 
server will operate, with a mean service time 
of 30 min and standard deviation of 15 min, 
regardless of whether the machine was actually 
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misaligned. This is all that is known of the 
service time distribution, but use can be made 
of the Pollaczek—Khintchine formula, which 
applies to single-channel queues with arbitrary 
service distributions: 


es (Ags) +6 
2(1 - p) 


(the notation is as in Section 11.10.3, with oy the 
standard deviation of service time). 

During the time that a machine is in the 
queue and being serviced, its lost production is 
costed at £200 per hour. In addition, if the 
machine is found to have been misaligned then 
its output for the previous several hours (given 
on average by the ARL) must be examined 
and if necessary rectified, at a cost of £10 per 
production hour. 

Find the total cost per hour for each option, 
and hence decide which control scheme should be 
implemented. 


Ns =p 


A transmission channel for binary data connects 

a source to a receiver. The source emits a 0 with 
probability a and a 1 with probability 1 — a, each 
symbol independent of every other. The noise in 
the channel causes some bits to be interpreted 
incorrectly. The probability that a bit will be 
inverted is p (whether a 0 or a 1, the channel is 
“symmetric’). 


10 


(a) Using Bayes’ theorem, express the four 
probabilities that the source symbol is a 0 or a 
1 given that the received symbol is a 0 ora 1. 

(b) Ifp is small and the receiver chooses to 
deliver whichever source symbol is the more 
likely given the received symbol, find the 
conditions on @ such that the source symbol 
is assumed to be the same as the received 
symbol. 


If discrete random variables X and Y can take 
possible values {u,,..., u,,} and {v,,...,v, 
respectively, with joint distribution P(w,, v;) 
(see Section 11.4.1), the mutual information 
between X and Y is defined as 

woe. In); 
I(X;Y) = 2 2 P(ug, Yj) log 5a u,)P(Yo oD aCe 
Show that for the binary symmetric transmission 
channel referred to in Review exercise 9, if X is 
the source symbol, Y the received symbol and 
C= 5 then 


I(X% Y)=1+plog,p + (1 —p) log, (1 —p) 


The interpretation of this quantity is that it 
measures (in ‘bits’) the average amount of 
information received for each bit of data 
transmitted. Show that /(X; Y) = 0 when p = } 
and that /(X; Y) > lasp—0andasp—- 1. 
Interpret this result. 
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6@) 20;[1 If -1y 232,1,-l[1 3 5B 2 Wo 0 If 

(b) 4,-1;[2 3), 0) -1] 

(@) 9,3,3:-1 2 IP 2 =i 4 a | 249.63: 2 ye 1 af 2 1 
3212 2 We 1 of -2 1 L = 1M Mmodal matrix 

() 14,7,-7;[2 6 3],[6 -3 2,B 2 -6 
(f) 2.1,-l:[-1 1 1,0 0 pn 2 77 
(2) 5,3,1;[2 3 -1J,[1 -1 oyf,fo -1 17 5 [0 0 I 
(h) 4,3,1;[2 -1 1,2 -1 oy, 4 1 27" 


on NM 
oN WN 


0 
0 
3 
7 (a) 5,[1 1. 1]'; 1 (repeated) with two linearly 
independent eigenvectors, e.g. [0 1 2]', 
flo -1y" 
(b) -1,[8 1  3]'; 2 (repeated) with one linearly 
independent eigenvector, e.g.[1 -1 0]' 
(c) 1,[4 1  -3]'; 2 (repeated) with one linearly 
independent engenvector, e.g.[3 1 —-2]' 27 A=-2:[0 1 1 
(d) 2,[2 1  2]": 1 (repeated) with two linearly A=4:[0 1 -l 
independent eigenvectors, e.g.[0 2 -—1]', 2 0 0 
[2 0 3] ap 
4 
0 


r- OS 


1 1 
2J=|0 1 

001 
o,f. 0 0 177 
oy,[6 -1 0 -6]" 
0 


81,[-3 1 1] 0 O 


0 
1 
92eg.f1 0 IJ, [0 1 17° 0 0 4 
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2 2 2 


29 (a) Positive-definite 
(b) Positive-semidefinite 


30 (a) 2a> 1, 


(c) Indefinite 
(b) 2b° < 6a-3 

31 Positive-semidefinite, eigenvalues 3, 3, 0 

32 k > 2; when k = 2 Q is positive-semidefinite 
33 a>2 

34 A>5 


36 (a) ; | wt 10 
2 3 5 7] 


Fo-3 {72 5 7! 
37 (a) = 7 b) J-1 - 
(a) 3 | (b) Gf-1 -3 5 
oe St 8 
47231 47342 47270 
38 | 47342 47195 47306 
47270 47306 47267 


39 (a) . : (b) ee | 
fe e ec =e. -e 


10 
0 
41 (a) ie a (b) 
-l 2t-1 


(d) x=}, y=-} 
47 (a) x=y=2 
F 4 5 1 6 
48 ©) 55 2 10 8 3 
3 0 3 3 


49 (a) ()x=y=l 
(b) ()x=y=l 
(c) @)x=y=l 
50 m=0.5,c=0.8 


(ii) x = y = 1.0909 
(ii) x = y = 1.4785 
(ii) x = y = 1.4998 


5t +5 
43 (a) 3,3 (b) Yes 
122 
er / 33 3 
44 (a) | 70 6/10 0 ] 2.12 
' 3.003 3 
we Sil © 310 Of] FF 
3. 3-3 
1 =I 13 
©) 5 4 8 
0 -10 
45 —_|}3 30 -17 6 -4 
141/18 9 9 30 27 
46 (a) 1 
[1 2 
3 9 Blfayo2 ola 2 
| 44/0 off 3 
229), 0 ol? # 


0 1 0 0 
51 (a) X=] Q 0 1|x+]o]u, 
l-4. -5 -4 1 
y=[1 0 O]x 
fo 1 0 0 0 
Hee |" q ; va ere al 
0 0 01 0 
0 -4 -2 0 5 
yell © 0 Ox 
Jo 1 40 0 
52 (a) x=] 0 0 1\x+]o}u, 
|-7 -5 -6 1 
y=[5 3 I1]x 
jo 1 0 0 
(b)x=/0 OO 1|x+]lolu, vy=—22 3 1)x 
|0 -3 -4 1 
Baan =R/L; =f, iL, 
53 X=)—-R,/L, —-(R,+Rs)/Ly —1/L,|¥ + | 1/L,|¥> 
1/C 1/C /C 0 
y=[0 R, O]x 
54 A possible model is 
B(M, + M)/MM, 1 0 0 
ya |-(KiM+KM,+KM)/MM, 0 1 0], 
—K,B/MM, 001 
| -K\K/MM, 0 0 0 
0 
0 
K,B/MM, | ° ge Bee 
K,K,/MM, 
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(RitRotRy) Re 


55 X, _ ac, ac, xy 
X> Re Ri + R3| | x 
ac, ac, 
R+R, 
ac, 
+ 
Ry 
ac, 
i Ri +R; 
- a a 
Ja) | Sig atery. “2a 
04 04 
R; 
+ u, 
R 
(Ry +R) 
a 


= RR; + (R, + R3)(Ro + Ry) 
—2.6 x 10°, -1.1 x 10? 


[ t 
56 7 : 


e'(1+2) te’ 
-te’ e(1-d 


57 


| y=e(1+2A) 
58 [el e(t+1)]" 
59 x, =2-4e%+3e%",x,=8e7-9e% 
60 x, =4te"+e7'—-e%7,x, =3e'-2e"%—4te" 


—Sthe‘+ We” 
61 x(t) = 


_8 -t 5 aot 
3-;e +3e 


62 x()=e'[3 2J'-e°[1 -2]" 


63 x(1) <4 14e'-4e% 
5 Te'+8e” 


64 x(f)=e"{(cos 2t— sin2A[2 1]" 
—(cos2r¢+sin2A[0 1]"} 


20 0 : 
65 zZ=|0 1 Olz+] Olu, 
0 0 -1 -4 
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676,114 Wo -1 
x, =4e% —3e', x, =e" + 3e! 


68 Same as Exercise 61 
69 Asymptotically stable 
70 Asymptotically stable 
71 a>0,b>0 


1.10 Review exercises 


1 (a) 5,2,-l:[1l 5 3],[8 2 1, 0 of 
(6) 3.210 2 W421 ofp o 
(c) 31,01 2 U5 0-5 1 17 

263,01 1 Wt 1 2, -1 of 

3 b=1,c=2;A=2,4, 1; 

i 2 ai 1 ay 

45.4 

5 (a) 4.56; [0.72 0.84 1]" (b) 1.75 
(c) (i) 1.19 (ii) 1.75 


60 <1 if 
743,12 2 1 2 14 2 of 
ee a a 


(64-20 =. 3-4 fo 1 2 af 
[0 0 1 -1,f0 0 0 Wh 
C-Ce+3Ce" — 3Ce*! 


— k 
10 (a) (i) - ; (ii) i 
—32 3 2°-1 #1 


-2t 
1 M1-e%) 
-2t 


0 e 


(b) 


11 e,=[1 0 Of,et=[2 1 Of,ef=[0 0 IF 
12 2,24 2; 1:0:-1, 1:—/2:1, 1:/2:1 
13 (a) Positive-semidefinite (b) Positive-definite 
(c) Indefinite (d) Negative-semidefinite 
(ce) Negative-definite 


14 1;3,[1 1 O]-1,f0 -1 17° 


08 06//5 0 0 os 
so [) ‘|| |} 08 0.6 0 


0.6 08|)/0 2.5 0 
-0.6 08 0 
_| 24 32 0.192 0.256 
b) —e = 
(b) 15) 18 24 0.144 0.192 
-20 15 -0.16 0.12 
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1 
16 (c) ahis[ 4 4 ee J 
; 2 4 18)-1 0 2 -2 
3 
[2 0 0 
17A=|/0 1 0|,5=[} 0 -#]', 
ad 0 +1 


c=[1 -4 -2]" 
The system is uncontrollable but observable; it is stable 


-2 1 -2 
1S M= | OQ =a 
-1 0 -l 


= -t 4127 ,,-2t _ 58,,-3t 414 _ 47 
l4e° +e rie sr aay 


_ -t 29-3 4 a 
x(t) Te‘ - Be +r +4 


t 


Fe 4 Wat 2 
Te +e ; 


19 (a) 6,[3 2 1733,[1 -1 O77 


(-3+3t)e"+3e" 

() 3] (1+3Ae" +20" 
~e +e 

20 x, = 2cost+2sint— 2cos2t 


x, = 2cost+2sint+ tcos2t 


CHAPTER 2 


Exercises 
1 X(0.3) = 0.985 05 
2 X(1.1) = 0.094 913 
3 X(1) = 1.1571 
4 X(0.5) = 2.1250 


5 X,(2) = 2.811489, X,(2) =2.819 944, 
x(t) = 2\(2 + t?)/\3 


6 X,(2) = 1.573065, X,(2) = 1.558541, 
x(t) = (1 +2Ind) 


7 X,(1.5) = 2.241257, X,(1.5) = 2.206 232, 
x(t) Inx(t) — x(t) = — 1.981214 


8 (a) X(0.5)=0.1238 — (b) X(1.2) = 1.3740 
9 X(0.5) = 1.7460 
10 (a) X(0.5)=0.7948 — (b) X(1) =- 1.3511 


14 X(0.5) = 0.1353 


15 (a) X(0.75) = 3.2345 
(b) X(2) = 2.2771 


16 (a) X),(2) = 2.242408, X,,(2) = 2.613 104 
Richardson extrapolation estimates the error 
as 0.123 565 so a step less than 0.0064 should 
be used. 

(b) Xo2(2) = 2.788 158, Xo,(2) = 2.863 456 
Richardson extrapolation estimates the error 
as 0.025 099 so a step less than 0.014 should 
be used. 

(c) X)4(2) = 2.884046, X),(2) = 2.897 402 
Richardson extrapolation estimates the error 
as 0.000 890 so a step less than 0.057 should 
be used. 

x(2) = 2.898 51 to 5 dp 


17 X(3) = 1.46647 


18 (a) dx/dt=v, x(0)=1 
du/dt = 4xt — 6(x? - t)v, v(0) =2 
(b) dx/dt=v, x(1)=2 
du/dt =—4(x? — #7), v(1) = 0.5 
(c) dx/dt=v, x(0)=0 
dv/dt = —sinv — 4x, v(0) =0 
(d) dx/dt=v, x(0)=1 
du/dt=w, v(0)=2 
dw/dt =e" + x°t-6ev—tw, w(0)=0 
(e) dx/dt=v, x(1)=1 
du/dt=w, v(1) =0 
dw/dt = sint—x*-tw, w(1)=-2 
(f) dx/dt=v, x(2)=0 
du/dt=w, v(2)=0 
dw/dt = (xt? + tw), w(2)=2 
(g) dx/dt=v, x(0)=0 
du/dt=w, v(0)=0 
dw/dt=u, w(0)=4 
du/dt=Int-—x?-xw, u(0)=-3 
(h) dx/dt=v, x(0)=a 
du/dt=w, v(0)=0 
dw/dt=u, w(0)=b 
du/dt = t? + 4t-5 + \(xf) -—v -(v- 1)tu, 
u(0) = 0 


19 (0.3) = 0.299 90 
20 X(0.3) = 0.299 64 
21 X(0.65) = —0.826 03 


22 X,,(1.6) = 1.220254, X,,(1.6) = 1.220055 
Richardson extrapolation estimates the error as 
0.000 013 so, to obtain an error less than 5 x 10°’, 
a step less than 0.088 should be used. 
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23° X (2.2) = 2.923 35036, Xo95(2.2) = 2.925 417 56 
Richardson extrapolation estimates the error as 
0.000 295 so, to obtain an error less than 5 x 10°, 
a step less than 0.0060 should be used. 


2.7 Review exercises 
1 X(0.5) = 1.548 860 
2 X(1.2) = 0.524 465 


3 X),(0.4) = 1.125583, Xo95(0.4) = 1.142 763 
Richardson extrapolation estimates the error as 
0.017 180 so, to obtain an error less than 5 x 10°, 
a step less than 0.0146 should be used. 


4 Xp o5(0.25) = 2.003 749, Xp.925(0.25) = 2.004 452 
Richardson extrapolation estimates the error as 
0.000 703 so, to obtain an error less than 5 x 10°, 
a step less than 0.0178 should be used. 


5 X,(1.2) = 2.374037, X,(1.2) = 2.374 148, 
X,(1.2) = 2.374 176 


6 X(1) = 5.194 323 accurate to 6dp. 


8 Xo.025(2) = 0.847 035, Xo.0125(2) = 0.844 066 
Richardson extrapolation estimates the error as 
0.002 969 so we have X(2) = 0.84. 


9 X(4) = 0.1458 (using step size 0.002) 
10 X(2.5) = —0.6532 (using step size 0.025) 


CHAPTER 3 


Exercises 
1 (a) Circles centre (0, 0), x* +? = 1+ e° 
(b) Straight lines through (—1, 0), y = (x + 1) tanC 
2 (a) Family of curves y? = 4x7(x — 1) + C 
(b) Family of curves y* = £x°(x°— 12)+C 
3 (a) z-xy=C 
(b) xy =In(C +z) 
4 (a) (Asec(t + B), tan(t + B), Ce’), curves on 
hyperbolic cylinders (x/A)? — y* = 1 
(b) Curves defined by the intersections of 
mutually orthogonal hyperbolic cylinders, 
ero-pyaexw-zak 
5 (a) f=yz—-2x,fp=azt+ 1 fi=xy-1, 
fez -2, hey Z, Se Vs Sip 0, Te Xs Jez 0 
(b) f.=2xy2?, fp = 2°72", f= 3x°y2", fy = 2yz’, 
To = 2xz*, f= 6xyz", foy =0, fi. = 3x°z*, f, = 6x7 yz 
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(c) f.=—yz/(x? +y),f, =2x/(x? +y’), 
f=tan (vy), fo = 2xyzl(x? +’, 
Soy = AP — XVM? + YY, fe = VI? +"), See = 0, 
fry = —2xyzl(x? +7), foe = x/(X? + y*) 


6 (a) 6f°(t? -— 1) + 8r+ 


(t-1)° 
(b) te*(cos 2t — sin 2t) + $e sin 2t 
F_ Ao Of sindcos@ , Af cos @ 
7 oy or ica al 00 r ’ ogrsin 0 
Ff A oogy — A Sin 
op be 


8 A/r+B 


15 9,v°-=u+2w 


17 w2UL+yd -—4uv)] 201+ yd - 4u’v’)] 
uj(l-4u’v) 9 vf —- 402)” 
oe 
VU = 4u?v?)) (1 = 4u’v’) 
(b) x*y? +ysin3x+c 
(d) zx-3xy+4y +e 


18 (a) xy? +x°vtx4+C 
(c) Not exact 


19 —-1, ysinx —xcosy + ‘(iy - 1) 


20 m=2 
8x° + 36x*y + 62x3y? + 63x7y9 + 54xy4 + 27> +. 


21 (36,9,12) (a) -%2 = (b) 39, 412, 3, 4) 


7 >B 
22 (a) (2x, 2y, -1) 
(b) (~yzM(x? + y*), xz/(x? + y’), tan'(y/x)) 
—xX—yrz 
(c) oe eye ws 
Grasp ot ae a ae ey) 
(d) (yzsina(x +y+z) + xyzcos n(x +y +z), 
xzsint(x +y +z) + mxyzcosm(x+y+z), 
xy sin@(x + y +z) + Mxyzcos t(x + y + z)) 


3 


23 3 
24 (5i+ 4j + 3k)/\50 


25 (a) rr (b) -r/r? 


26 b=x°y+2x+zy 


27 -9j + 3k, * 
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28 54°25’ 75 1 (6m — 20) 
29 (a) x+ 2y+3z=6,x-1=3(y-1)=3@- 77 in+2/n-1 

(b) 2x + 2y —3z=-3, 

1-1) = (y-2)=4G-2) as 

(c) 2x+4y—2=6, }@-1)=!(y-2)=4-2 1D 5 
31 (a) 6xy —(b) 4 80 0 
32 -61 81 Sa(1— 5m) 
33 a,a,3a 83 Bq 
35 -13 84 (a) “P(b) Gt 
38 (9, 6xz -— 1, 0) 85 (a) a (b) 0 
40 x? ty? +27 +xyz 87 (a) Bn (b) nc) Ee 
42 a=2,b=2,c=3; b= 2x*y + 2z*x + 3zy + const 88 24n 
43 \llrads" 90 90 
44 d=-a,c=b 91 0 
47 (a) 2y*z3 + 2x23 + 6x°y*z 92 (a) 2 (b) # 

(b) 2v(1 + z)i+ 2 4+-xz-z)j + 2v(x- Dk 

(c) 2yzi + 2(x —z)j + 2yxk 94 3n-2 
56 156 95 ix 
57 -} 96 L 
58 '¢ 97 ara 
59 (a2) (by 10. (c) 8 98 —_ 
60 10.54+ 47 99 (1-e')/6 
61 (a) 16 (b) 16 100 2 2 

(c) Not necessarily. The value has to be the same for 

all possible paths. 101 5x 

62 35 102 tna‘ 
63 —2i-2j+2k 103 3 
64 4n(7i + 37) 104 16x 
65 (a) 24 (b) 76 ~—s (c) 16 105 84x 
66 $In2 109 2nab 
67 - 110 16x 
68 (a) (In2)tan'(+) (b) + (c) 1 
69 -8/(3n2) 3.7 Review exercises 
70 (a) 4(2-1) — b) (1 — #8)? = 13H? ia aad 
1 %\2-1) Tie -yxth’—ty +e 
7A 8 (a) (vb) 4 
74 2a(1 - +n) aS 
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3: 
13 2ha?| tn - sin'(£ J = Del 4 tank) 
3 2 a 3 3a a 


I=\(@-) 


14 ¥a° 
15 nqyrl/4El 
16 ; 
17 0 
19 0 
20 = 
CHAPTER 4 
Exercises 
1@) y=ix+} (b) y=hr-} 
22z=2,5% 
3 u=6v 
6 Semi-infinite strip v > 0, |u| < 1 
7 (a) u=v/3-4 
(b) v=-u 3 


(c) @t+1Pt+(v-\3P =4 
(d) wt+v?=8 
8 (a) a= (-2+j), B= 31 +2)) 
(b) u+2v0 <3 
(c) (5u—3) + (Sv—-6) < 20 
(d) 5(1 +3)) 
9 Interior of circle, centre (0, —1/2c), radius 1/2c; 


half-plane v < 0; region outside the circle, centre 
(0, —1/2c), radius 1/2c 


10 Circle, centre (+, —3), radius 7 


11 Re(w) = 1/2a, half-plane Re(w) > 1/2a 


ie wees, 
z~J 
Re(z) = const() to circles 
2 
v+(v-—E} = : > plus v=—-1 (k= 1) 
l-k (1 -—k) 
2 

Im(z) = const(/) to circles (u + ) +(v +l) = 1 

l 


plus u=0 (/=0) 
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13 (a) 1+j,j,° 
(b) |w| > \2 
(c) v=0,(@u-1P+uv=1 
(d) +2'4 gins 


14 Segment of the imaginary axis |v| => 1 


15 (a) Upper segment of the circle, centre (3, -3 ), radius 
1/5, cut off by the line uw —3v=1 


16 Circle, centre (2, 0), radius 4 


17 %=),=7 

18 Jw-1|<1; w-3| > 

19 w= 6° 2— 6. where 0, is any real number 
zyz—1 


20 Region enclosed between the inverted parabola 
v = 2 —(u’/8) and the real axis 


21 uw=0, 2mu =(1 —m’)v 


23 u=xt+ 


x 2 
5 U=y- = ;v = 0; ellipses, 
w+v =r andx’+y’ =r’, r large 


24 (a) e(z+ 1) (b) 4 cos 4z (c) not analytic 
(d) —2 sin 2z 


268 a=-i,b=1 
w=z° + jz’, dw/dz = 2(1 +j)z 


26 v=2y4x?-y? 
27 e(xsiny + y cosy), ze 
28 cosxsinhy, sinz 
29 (a) x - 6x’? +)* = B 
(b) 2e* siny+x*-y’=B 
30 (a) (x? —y’) cos 2x — 2xy sin 2y 
+ j[2xy cos 2x + (x? — y’) sin 2y] 
(b) sin 2x cosh 2y + j cos 2x sinh 2y 
31 u=cos {27 fx? +y°- 14 [G7 +9? - 17 + 4y7]}} 
y= sinh {30 +y’-D+hHIG?+y’- 1) +4] 
33 (a) 0 
(b) 3,4 
(5 gl HIND), {O19 
34 z=4j 
35 (a) region outside unit circle 


(b) 1S wv4+v<e,0<vS<utanl 
(c) outside unit circle, wu and v of opposite sign 
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36 YA whole w plane v 
oom 
[| | 
GHG! 
Ee) = 
LE 2n x 
— 5 


x =k — hyperbola 
y=k-— ellipse 

a+b 
b 


37 4a, ellipse centred at origin, semi axes are 


and aa 


38 (a) jre= je-2 +i +. 
(b) 
(c) 


39 (a) 1—227+3z4-42°+... 
(b) 1-—3z7+ 6z*- 10z°+... 
2 3. 
40 (a) 1-4(z- 1) +z - 1)? - Le - 1)°;2 
ne «4 -\6 
(b) 3-42-21) +@@- 21) -gy@—2)) 32 


(c) -3j+4(1+j)(@-1-j)+3(@-1-j) 
+i(j-I@-1-j)°; 2 
41 l-z+z3+... 
42 1,1, \5; fis singular at z =j 


43 z+iz +27° +. 3 4n 


44 (a) 1424324 427+...(0<|z|<D 
Z 


14 


(z-1)? z-1 
(O0< |z-1/< 1} 


(b) +1-(z-D+(z-D-... 


45 (a) ...+ 


(b) nee aera ! 


3lz Sz" 
(c) asin + + zf’(a) 4 


z fa) ! 


46 (a) bz + 32° +z * 4 zt 


(b) 2a 1 i 1 - =2 des 
Zz 2 2 4 8 
(c) 14342414 
ZZ 2£ 
(dy) +++ 4+ : aa 
z-1 (@-l) ¢-) 


sv 


47 


48 


50 


51 


52 


53 —4 


54 
56 
57 
58 


Gf Hote heady 27 fea ar" 
z-2 


+(z-2)'- 


(a) z=0, double pole 

(b) z=j, simple pole; z = —j, double pole 

(c) z=+1, +], simple poles 

(d) z=jnn (n an integer), simple poles 

(e) z=+)7, simple poles 

(f) z= 1, essential singularity 

(g) Simple zero at z = 1 and simple poles at z = +j 

(h) Simple zero at z =—j, simple pole at z= 3 anda 
pole of order 3 at z = —2 

(i) Simple poles at z= 2+), 2—j anda pole of order 
2atz=0 


5 
(a) a = a arte -... (removable singularity) 
3 =| is 
(b) +4 Liga hol. .. (pole of order 3) 
z z 2) 3! 4! 5! 
(c) ea ee (essential singularity) 
z 2! 4l2° 


6 


(d) tan '2+ 2 = 552 +... (analytic point) 


(a) Simple poles at z= —1, 2; residues +, 2 


(b) Simple pole at z = 1, double pole at z = 0; residues 
-1,1 

(c) Simple poles at z = 1, 3), 
£G-j), £6 +i) 

(d) Simple poles at z=0, 2j, -2); residues —+, 
rale yal 

(e) Pola of order 5 at z = 1, residue 19 

(f) Pole of order 2 at z= 1, residue 4 

(g) Simple pele 8 z=-3, double pole at z = 1; 
residues -i,4 i 

(h) Simple poles at z = 0, —2, 


—3j; residues $, 


1; residues 3, -3, 1 
(a) 1 (simple pole) 
(b) —5@ +jy3) sin[ZC + jy3)] 


(c) 4 +j) 2 
(d) -1 


(simple pole) 
(simple pole) 

(©) -i; 
(b) -3 


(simple pole) (double pole) 


(a) -} 


(c) e” 


(triple pole) (double pole) 


(double pole) 


- is , all cases 
0, all cases 
(a) 0, (b) 2nj 
nj, $j 


4n(9+j2),0 
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37] 
59 (a) —3T) (b) 0 xeko(u- Vaot=— 3 
60 (a) 0 (b) 2nj a wa 1) 
; as ~17+( +t at 
61 (a) —4nj —(b) 2nj sai lL ? 
62 z=), -isz=-i, 3j 2 =j\6, 2j 6; Fixed points: 1 + 2 
Z=—j\6, —73j y6 6 Fixed points z = +,/2/2 
(a) 0, (b) 2, (c) 0 r=l>u=0 
63 (a) 0 (b) 0 7 u=x? — 3xy’,v=3x°y-y? 
64 (a) 2nj, nj 8 (zsinz) v=ysinxcoshy + xcosx sinhy 
(b) £1(25 - j39) 9welz 
c) 0, nj, -—2nj 2 7 
©) en aoe : 10 Ellipse is given by —— + ——t =1 
(d) 0-751), -31j (R+a/4ky (R-a/4k) 
65 (a) 2n/3 (bo) nm ~— ) Sem) St Ie De ee ee eee 
(e) bx (f) In (g) x (h) 1/22 1—227° + 32°-—427° +... 
(i) im (j) m1 —3/)5) 12 (a) 1-224 227-22';1 


3 (b) $-}2-N+h@-lP-ie-I"; 2 
il (c) 1 +5) + Liz -j)- 40 +) @- f° -He-5)?s 2 


67 (a) (0, 0), (0, 1), (0, 7), (7, 9) 13 1, 1,1, 4/5, 2,2 respectively 


(b) v=0  (c) u=0 m2 
8 He. ater 14 (a) toztz—2°+...0 Pa ee 
W = 22 -j2° 


(b) L-(z-1)+3(z-1)'+... (lz-1]<1) 


70 (a) (0, 0), (1, 0), (-1, 0) 


15 (a) Taylor series 
(b) u=0 (c) v=0 (a) : 


(b) and (c) are essential singularities, the principal 
parts are infinite 


4.9 Review exercises 16 (a) (e cos2y- 1) + jhe™ sin 2y 
1 (a) 3j (b) 7 +4 (©) 1 (d) j2 (b) cos 2x cosh 2y — j sin 2x sinh 2y 
(c) 
2 (a) y= 2x gives 3ut+v=3,u+2v=3 and3v-—u=1 xsinx coshy + ycosxsinhy +j(xcosxsinhy - ysinx coshy) 
respectively x+y 
(b) x+y=1 givesv=1,v—u=3 and u= 1 respectively @ tan x(1 - tanh’) ) +jtanh y(1 + tan?x) 
2 2 

3 (a) a= -1(34+j4), B=3+j 1 + tan’x tanh’ y 

(b) 13 S 3u+4v 17 (a) Conformal (b) j,-l1-—j (c) +0.465, +j0.465 


(c) lw-3-j]S1 @ X(7-)) 18 


4 (a) Wt+v+u-v=0 (b) u=3v 
(c) W+v+u-—2v=0 (d) 4’ +v°) =u 


whole w plane 


5 Left hand Right hand 
v 


y z plane 


vy 

2 z - 2 = 
cos k  sin’k 
2 
| 


+ — 
cosh’! sinh’/ 


x =k — hyperbolas, 


y=1- ellipses, 
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19 (a) Simple pole at z=0 

(b) Double poles at z = 2, 2e*"!?, 2e* 

(c) Simple poles at z= +1, +j, removable singularity 
atz=-1 

(d) Simple poles at z = }(2n + Inj 
(wn =0, +1, +2,...) 

(e) No singularities in finite plane (entire) 

(f) Essential singularity at z= 0 

(g) Essential (non-isolated) singularity at z = 0 


20 (a) 2e° (b) 0 (c) 0 (d) 0 
21 Zeros: +1, -2+3j/11 


2052 2: 
Poles: 0, eni4 ens ens e/ni/4 
: : 64+3\2_.; 
Residues (respectively) —5, a ak 
OSE a. ooo 5, 6+3 Lane 
4 4 4 
22 —204 — 324) 
23 (a) —21j (b) 0 (c) (i) 0, (ii) 3mj_~— (dd) *0~,0 
: 4n. 
1 fy 72,2 
(e) f) jg—ad 
; lln 19% 
24 (a) Zn (b) i\2 (c) v9 (d) cis 
CHAPTER 5 
Exercises 


1 (a) S45, Re(s) > 2) 2 Re(s) > 0 
S 


Se 


(c) ,Re(s)>0 ) ee Re(s) = -1 


s° (s+1) 
2(a)5 (b)3 (0 @3 2 
(f) 0 (g) 0 (h) 0 G2 (j) 3 


3 (a) 823 Re(s) > 0 
S 
(b) 2-8 Reis) > 0 
s gs +9 
(c) 3822448 Re(s) > 0 
Ss gs +4 
(d) =—, Re(s) > 3 
ve 
(e) 5 Re(s) > 2 
ve 
Gy 243-2 Res 0 
st+2 5s 44 
(g) 33 Re(s) > —2 
(s+2) 


(i). <=! =» Rete S 3 

s +6s+ 13 
(i) z Re(s) > -4 

(s+4) 

2 3 
(j) 307 Os 4s ~ 25" Re(s) >0 
(k) At Re(s) > 0 
+9 

a) S=4, Res) > 0 

(s +4) 

2 

(my 82 => Re(s) > 0 

(s +9) 
(n) 2_—3s_ as ,Re(s) > 0 

ss +16. 
(0) 2 -+—St! 43 Re(s) > 0 

(S#2) gs +25+5 5 
(a) e'-e") (b) -eT +2” 

-3t . 

(ey §-5f=Ge (d) 2 cos 2t + 3 sin 2t 
(e) g(4t- sin4r) (f) e (cost + 6sin#) 


(g) }1-e° cos 2t+3e” sin 2t) 

(h) e'—e'+2te* 

(i) e “(cos 2t+3sin2A) (j) te'-3e%4+4 
(k) -2e7°' +2 cos( 24) - \$ sin(\22) 

(l) $e'-2e (cos t- 3 sins) 


(m) e“(cos 2t — sin 2¢) (n) tem =9¢° ‘43 
-2t 


u e 


3 er 


(0) -e'+3e%-te 
(p) 4-3cost+4 cos 3t 
(q) 9e"'- e "17 cos(t \3t) - /3sin(3/30)] 


(rt) fe'-te Let (cos3t+3 sin3t) 


(a) x(t)=e"% +e" 

(b) x(t) =8 e* — 3(cos 2¢ +? sin 22) 

(c) x(t)= 11 -e cos2t-}esin 21) 

(d) y(t)=4(12 e+ 30te- 12 cos21¢+ 16 sin 2) 
(e) x(H= -le "pte" 44 e 

(ff) x()=e ee ne 

(g) x(t)=Be'- be" + 4e“(cos 27 - 3 sin2/) 
(h) pee eae e heats |21) + \3 sin(/2¢) ] 


(i) x(h=G+ine+ife"+i- 


(j) eer 
(k) x(t) =te-!cos4t 

(l) y(t) =e7%+ 2te?” 

(m) x(t) =34+$t-e' + 5e%-2e° 


1 1 {2 
ge tat 


t 


i 
(n) x(t) =2e Megat nats 39 COS3t 


oe 
- 55 sin 3¢ 
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6 (a) x(t)=h(Be"—Hel-e™), y(t) = 3(3e"- e') 
(b) x(¢) =5sint+S5cost—e'—e"%-3 
y(t) = 2e'— S5sint+ ec -3 
(c) ee 
1 


y(t)=—2 {sin +3 cost-5e 


(d) x(t) = je aed y(t) =-l+ze'+3e 
(e) x(t) =2e'+sint—2cost 
y(t) = cost — 2sint—2e' 
(f) x<Q=-3+e+3e" 
yUyst=1=Fe +26 
(g) x()=2t-e' +e, yp(th=t-4+3e'+he 
(h) x(t) = 3 cost + cos(\3A) 
y(t) = 3 cost — cos( 32) 
(i) x(t) = cos(\5t) +3 cos(/6r) 
y(t) = } cos( 752) - t cos( /6¢) 
(j) x(t) = ;e'+2 cos2¢+ 5 sin2r 


-t/3 


2t 


y(t) = Fe’ -2cos2r-}sin2¢ 


Ace es 
(s? + 10°)(s + 100) 

L(s)= >. 
(s?-+10*)(s + 100)” 


i(t) = E(u + gt + Z cos 1002) 


9 i,(t)=20\2 e“ sin($/72) 


10 x,(¢)=—Acos(/34) - 3 cos( 132) 
x,(t)=-4 5 COSC, [3t) +3 cos(137), y3, \13 


13 f(t) = tH(t) — tH(t- 1) 


14 (a) f() =30 - B(t— 4) + 22(¢ - 4) + 43] A(t — 4) 
[2(t — 6) + 4]A(t — 6) 
6 6 22 , 43) -4s 2,4) -6s 
F(s)==-(|=-4+>4+= es (eae 
(s) 2 Gas, (4+2)e 
(b) f()=t-2(t- 1)A(t- 1) + (t- 2)A(t- 2) 


F(s)=4-20%+4 e* 
Ss Ss 


15 (a) 1(¢- 5) ee“ H(t- 5) 
(b) 3[e? - e*" JA(t - 2) 
(c) [t—cos(t— 1) — sin(t- lI JA(t- 1) 
(d) yg" { 3 cos[4/3(t- m)] 
+ sin [Ly3(¢-m)]}H(t-m) 
(e) H(t- 2m) cos5t 
(f) [t—cos(t— 1) — sin(t— 1I)JA(t- 1) 


16 x(t) =e" +(t- D1 -A(t- 1] 


ANSWERS TO EXERCISES 1005 


17 x(t)=2e"’ cos($/3t) +¢- 1-2H(t- 1) 
{t-2+e” fcos[t/3(t- 1)] 
- /tsin[4/3(¢- 1)]}} 
+H(t-2){t-3+e0” 
{cos[ +/3(t- 2)] - /sin[$/3(t- 2)1}} 


2t 


18 x(t)=e'+4(sint-3cost+4e"e 
—5e™ e")A(t- 5m) 
19 f(t) =3 + 2(t- 4)A(t - 4) 


F(s)=3 is e* 


x(t) =3 oa 2[t—4-sin(t—4)]AH(t- 4) 


20 6,(t)=3(1 - e ‘cost- 3e° sins) 
— 311 - ee cos(t- a) 
3a -3t 


—3e"e sin(t- a)|H(t- a) 


21 @()=5(3 - 2t- 3e “- 10re “) 
+ 5[2¢-34(2t- le A(t - 1) 


25 (a) 26(t) + 9e7 — 19e* 

(b) d(t) - 3sin2¢ 

(c) 6(t)- e“(2cos2t+}sin2r) 
26 (a) x(t)=(L-2e%+le%) 
+(e ee 2) eke t= 2 2) 

(b) x(t) =} ee H(t - 2m) sin2t 

(c) x(t) =5e*%—4e%4 (6%) —e *™) A(t 3) 
27 (a) f' (ON =2'(t) — 430(t — 4) — 46(t - 6) 
6t (0<t<4) 


g(th=\2 (4<1<6) 
0 (t= 6) 
i @=sf<1) 
(b) g’(t)=4-1 (1 <t< 2) 
0 @=2) 
(c) f(t) = 9) + 58(t) — 65(t — 2) + 158(t — 4) 
2 (0< 7 = 2) 
g(t) =4-3 (= f= 4) 
2t-1 (t=4) 


28 x(t)=—Be +e" tre" 
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Jk. -yt: 2_ 1 R meee 51 e*%-e% 
eg ee a eae x(t)= SA[1- 40% 430% - (1-48 
-4(-T) = 
(y= etn cos nt- Ll sin nt) ae = TN 
" 52 e*sint, {fl - e (cost +2sin?)] 
1 4 3 
31 = ——[2Mx'/1+ 8W(x - 11) A(x - 41 r 7 r 4 
ai aE : eee 53 ally ete 2a)" 
—4(M + W)x? + (2M + 3W)Px] a} | 3-1] |x ; Xp 
2 2, 2 3 
— WX = x)x W(X) = X))X G(s) = 12s +59 
he a 6EI (s+2)(s+4) 
w AKA ge aE r. 
OM ees sa] = [7 ‘ : Hae =i af 
— (x = x)'A(x - x)] | [-6 0} | l *2 
= wl*/8EI rc 
‘aoe 0 1 0] fo 
33 y(x)= PEL te ~ Lox by W(x ~ b) ~ 33°] 3S faye) OO TRE Ole: 
at =e 6); 5 
2 = 
~ 2% (3b- x) (0<x <b) i a a 
° 0 1 0 0 
Wh = _ 
EEX) (b<x<l) (b)¥=/0 O 1/xt+/Olu, yvy=[2 3 I]x 
le 3 44 1 
34 (a) a -St+he'+ We” 
gs +2s4+5 57 x(t) = i ea 
(b) s°+2s+5=0, order 2 are 
—| +72: 2 
(c) Poles —1 +j2; zero —$ 58 x, <1) =2e2-e 
2 S 
35 —STSSFO 45574179 + 13 =0 Ate*+e2! 
s +5s°+17s +13 59 x(t) = 4. de es 
order 3, zeros —3, —2, poles —1, —-2 + j3 —4te -2e +2e | 
36 (a) Marginally stable (b) Unstable | _1 0 0 1] 
tabl tabl tabl 
(c) Stable (d) Stable (e) Unstable Ee |. .o. wlerl aly popes dep 
37 (a) Unstable 0 0 -3 1 
(b) Stable L @:\ 
(c) Marginally stable The system is stable, controllable and observable 
(d) Stable : 
(e) Stable 0 OO O ; 
2 61 z=/0 -1 Olzt)/—tlu, y=[5 3 15]z 
40 K >? a ‘ 
41 (a) 3e¢"-3e% (b) te “‘sin3¢ L ~ “ 
(c) 2(e" -e”) (d) } e”sin3t The system is marginally stable, controllable and 
observable 
ee 62 15 5 9 ,-2t 6 —t 
(s + 1)(s +2)(s +4) 4 3it ge" —6e 
47 55 a 0 1 0 of} jo o 
49 (a) L[2-e "(9F + 61 +2)] 63 [2] = |-1 -1 9 1] P{ fio “ 
(b) ble ™"(5t+2) +e%(5t- 2)] he 0 0 0 = 1/)%3} 10 OF [M2 
() £(4t-1+e") oe Oat Sie, 4 
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x} 
vy) _ fi 0 0 0 f% 
a) 0 0 1 Of }%3 
x4 
| 2 
G(s) = ! : s ts+l1 Ss 
(stIy(s +1] 5 sts+] 
Sis) f-a =a -a]fm] ft ap, 
64 (a) |x,| = |-1 -3 -3}}%2;/+]-1 1 A 
| f-d. =3° 6) les! [a a) 


_ 1]-2s(2s+3) 2s(2s+3) 
(b) Gis) = Al : : } 
-S AY 
A=s°+10s°+165+6 


(c) y(t) = 1 + 0.5788" — 1.8240" + 0.24601 
yet) = 0.177e*™ + 0.27265 — 0,.449e1 


65 u(t) = [-# = ]x( a Uext 


66 u(t) = [-2 — ]x() a Yoxt 


67 u(t) = [#2] x0) + Hew 
u(t) =[-31  —11) x(t) + Uext 


_ 0 1 
69 M= ki fk me i 
1 -l 5 


Nu 


5.10 Review exercises 


1 (a) x(f) =cost+ sint— e(cost + 3 sin#) 
(b) x()=-3 + Be'+ Be” 


2 (a) e'- te” - $e "(cos t+ sin f) 
(b) i(f)=2e'-2e* 


+ V[e"-te 


or Le “(cos t+sin f)] 


3 x(t) =-t+5sint—2sin2t, 
y(t) = 1 — 2 cost+ cos 2t 
4 ‘(cost +2 sin?) 
e'[(xo - $) cost + (x, +xq - 3) sin¢] 
\, 63.4° lag 
s.cos@- @ sin 


6 (a) (i) $0. 


s+@o 
.. § Sind+ @(cos¢+sin@) 
(i1) 


s +2@s+20 


10 x(t)= Fe" +58 


18 i(t) = [e 


ANSWERS TO EXERCISES 1007 


(b) 4 (cos 2¢ + 2sin 2t) + Fe “(39 cos 2¢ + 47 sin 21) 


7 (a) e~(cos 3t— 2 sin 32) 
(b) (tf) =2+2sint—5e* 


8 x(t) =e +sint, y(t) =e“ — cost 


9 q(t) = (5 et 7 2 ey 
— <,(3 cos 100¢ - sin 1007), 
current leads by approximately 18.5° 


-t/5 1_.-2t 
1212 e€ + 3 


— ¢:(76 cos 21 - 48 sin 27) 


11 (a) 0=4(4e "4+ 10te  - 4cos 2+ 3sin 22) 


(b) i= He" +6e"), = Hee") 


12 i=2U -e "(cos nt + sin nt)] 


-RUL —-3RU/L 
13 i, _E(4-3e =e) , i> EBR 


6R 


14 x,(H)= ‘Tsint - 2sin2t+ 3 sin(/3¢)] 


x,(t) = {[sint + sin2t- 3 sin(/34)] 


15 (a) (i) e“(cos3¢+ sin 3f) 


(ii) e' — e* + 2te’ 
(b) p()=fe"(8 + 122+ 8°) 


16 (a) 3e”sin2¢ 


) ni 
2 2 
Ks(s°+2Ks+n°) 


Ct 


jeter 
K 


17 (a) (ii) eo“ [cos 2(t- @) - $sin2(t- a) ]H(t- @) 


=F : . 
(b) y(t) = le ‘(cos 2t - tsin2r) + 2 sin t - cost] 
+ +fe (cos 2t - isin 2t) + cost 
— 2sin t] H(t — 7) 
-40t —40(1-T/2) 
-2H(1-1T)e 
+2H(t-T)e 
-40(t-3 7/2) 
-2H(t- 37) $2] . 
Yes, since time constant is large compared with T 


19 e‘sinz¢, itl -e(cost+sin/)] 


d 


4 
20 EI* = 12 + 12H(x- 4) - Rd(x- 4), 
d 


x 
yO) = ¥'(0) = (4) = y(5) = yO) = 0 
1x4 4.25x°+9x° (0 <x <4) 
yx) = 1.4 3 2. 1 4 3 
ix* — 4.25x° + 9x? + L(x - 4)* - 7.75(x - 4) 


(4<x <5) 


25.5kN, 18kNm 
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21 (a) f(t) =H(t- 1)- H(t-2) 
x() = H(t- I) — ee“) - H(t 2) 
(b) 0, E/R 


23 (a) t-24+(t+2)e" 
(b) y=t+2—2e'+2re', (=tety, 


et) 


: 
24 Ely =-2Whe + Wx - EGA ie -1) 


81 


4 
EI = =-W68(x -1) — w[H(x) - H(x-D] 
x 


25 (a) x)= 
Lp +e PT 3 sin(4 (32) - cos(4 34) ]H(t - a)} 
1 


26 (a) No 0) === = 
s +2s+(K- 3) 


(d) K>3 


27 (a) 4 (b) 


28 (c) 4e°% —3e", wr) = 1 (t= 0) 


= 
29 x(t) = e ‘sint 
1-e ‘(cost+ sin?) 
A(s) = —st2 e“(cost + sin f) 
(s+1) +1 


30-1,-1,-2: 1. 0-17, =i 
u(t) = —6{x,(t) +x,(¢)} 


~ ER 
a 


_ st3 
= 1 


oy,f0 0 17 


3 


— 


K 


) ——————— 
s+(1+KK,)s+K 
(c) K= 12.5, K, =0.178 


33 (a) K,=M,@ 


34 (b) Unstable 
(d) -8 dB, 24° 
(e) K=10°, t, =10°°, 7=107, ,=4x 10% 
(f) s° +36 x 10%s? + 285 x 10"s 
+25 x 10'8(1 + 107B) =0 


32 (a 


(d) 0.655, 2.485, 1.868 


(c) B=2.5 x 10°, 92 dB 


CHAPTER 6 


Exercises 


4z 
4z—1’ 


1 (a) IzI> 0 @ 4 lz1>3 


4 


1 


os 


12 


13 


14 


15 


16 


Zz -Z 
Ss >2 d —., > 2 
Sry Ll @ —>5° 12! 


Zz 


(e) 345, |z|> 1 
(z- I) 
-2@kT Z 
. ° -20T 
ra 
1_22 __ 2 
z2z-1 27 (27-1) 
5z Zz 
b 
ee) 
22 2z 
Qe-1 roi) 


(a) fen oS Zz 
zZ-¢e 


(b) {sinkT}o 


-4T 


z sin T 
z -2zcosT+1 
(c) {cos 2kT} eet 

z -2zcos2T+1 

@t cy © @ 
() j§ — (£) Giy2)! 
(g) 0(k=0), 1 (kK > 0) 
(h) 1 (k= 0), (-1)"! (k > 0) 


(a) [1 - (-2)'] (b) 4[3*-(-)'] 

(c) $+4(-4)* (d) 3(4)°+3(-1)" 

(ce) sin tkn (f) 2‘ sin tkn 

(g) $k+1(1 - 3°) (h) +2)! cos(tk - 3m) 


(d) 4-1)" 


(a) {0,1,0,0,0,0,0,2} 

(b) {1,0,3,0,0,0,0,0, 0, -2} 

(c) {5,0,0,1,3}  (d) £0,0,1,1}+{(-4)} 
(c) 1(k=0), (k= 1) (k= 2) -(-D' (k= 3) 


0 k=0 
©) | (k = 0) 
$-9742"" S11) 
0 (k=0) 
®) i (k = 1) 


1 E 1 Wry os 
Ver2 + Ver = Xk View + GV ~— 5 Ve = Xe 


(a) yak (6) ye= 799") + H(-L" 
(c) 2*"sintkn = (d) 2(-4* +3" 


@) n= %-b'- 20)"+} 
(b) y= 43") - 6(2') +8 
(c) y,=2(3") — 22") + 405" 
(d) y,=—-2(/3)”” sin ‘nnt1 
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(©) y,=—2(-})" + 2(2)"- 20-1 
(f) y,=-}2"+(-2)"]4 1-2 


17 (b) 7, £4841 
18 y,=2°-103')+! 
19 Ask, I, > 2G as a damped oscillation 


21 (a) —_1— 
z-3z+2 


z-1 


23 (a) 4{(-1)'- (-4) (b) 2(3") sin 4(k+ 1) 


(c) 2(0.4)"+ 4(-0.2)' (dy 4" + 2% 
‘4 i | (k = 0) 
rl REY) 
f (k = 0) 
2" (k= 1) 


25 (a), (b) and (c) are stable; (d) is unstable; 
(e) is marginally stable 


26 2-(1)" 


28 y,=—4(4)" + 2(4)" +2(3)" 
k k = 
ee Seer et 
4\4 2 4 |-4 2 


) cue aca | 


=. <I 1 1 


©) cn! “| 
0 1 


31 x(k) = 5\(coskO+ sink@), y(k) = 5‘(2 cos k6), 


=3 
cos 0= —= 


ANSWERS TO EXERCISES 1009 


25_ 11(_0,2)* + 2(-0.8)' 
32 x(k) = 
2 - (3,4/6)(—0.2)' - (17.6/9)(-0.8)" 


38 in (set) (58) 


2 2 
34 [el +17) _ | 41 -e°")| xi (kT) 
mUk+ITY |g  e2t | 2(kT) 
1 ly PF 
, [32+ ale Dkr) 
Li =e") 


Prae Bs ae : hs are " Shaan 
wlk+tT] |-7 1-Tlxkn| {7 
WAT) = [1 0] -x(kT) 

(b) x[(k + 1)T] = Gx(kT) + Hu(kT) 
WAT) =[1 0] -x(kT) 
_ppf COS(ZT) + 4sin( $7) 
G=e" ’ 
-4sin(37) 
aon ST) 
eee 
i - €™ cos(2T)- 46” man) 


-T/2 + 
3e sin(}7) 


37 (a) x(k+1)= fee ho a a 


0.632 1 0.368k, -1 
(b) x(k-+1) = [9-368 -0.1185 x,(k) 
0.632 1 [x2(k) 


, [0.1185 0 k, 
0.069 -1)]1.1x,(0) 


(c) x,(t) =x,(0)[1.1 — 2.15e!™ + 2.05344 
x(t) = k, + x,(0)[-5.867 + 8.6e! — 2.71e3""] 


38 q form: 
(Aq + Bq+C)y,=A(¢ + 2q + lu, 
6 form: 
[AAS + (2A4 + AB)S+ (4 +B +C)]y, 
= (4 +405 + 28)u, 
A=2N°+6A+4 
B=4A°-8 
C=2N -6A+4 


1 


si +29’ +2541 
[(A? + 4A? + 8A + 8)5 + (6A? + 16A + 16)8 
+ (12A + 16)54 8], = (2 + T8)'u, 


39 
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2 
41 12 =z) 
(12+ 5A)z + (8A - 12)z- 
12yd+Ay) 
A(12 +5A)/V +(8A- 12)y+12 


6.12 Review exercises 
4 342k 
5 1+ 4(-2)'- i(-1)" 


7 Z = Zz 
(z-e)” z-e7" 
1 n n 
8 (a) {aa -6| 


(b) (i) 3°'k (ii) 2. $ sin tka 
9 3-4(-1)'- 2! 
10 (-1)‘ 
13 54[2 - 2(5)"- n(3)"'] 
17[1 3 1.B 2 1.0 017 
== 3(2 43 
x(k) = 1-2" 
els )+4 
18 D(z)=*3— m-|' 3) 


z+4z-5 


+s -1 1 sss T 1 
(ii) M, = Gi) » =[0 —3] 
2 


0 L 
GAT < 2) re[ ‘ 
1 2 0 
a=-5, B=4 


CHAPTER 7 


Exercises 


1 (a) f= —7n-= = cos(2n = It 
T 


+> 3sin(2n - 1)t_ sin 2nt 
-_ 2n-1 2n 


(b) fio=in+2y cos(2n - Ur y ae 
Tt (2n- 1) 


n 


© Ay=2 5 


n 


(d) f= 


(e) f= 
_ re — cos(2n - I)t 
(2n- 1) 


(g) ji > efaei yy neat 
n n 


(h) f(t) = ($x + sinh) 


WIP a 8 I 


(f) A) 


n=1 n=1 


— — 1, Cl sinh ensue 
a n+l 
ren sinh & sin nt 
mean +1 
a 


2 fai +4 


= n 


Taking ¢ = 7 gives the required result. 


a 4=0l4-2 >) am 
2 wt (Qn- 


10 cos 2nt 
4 f=2+ > sin t - — 
2 ae ye: -1 


n=l 


5 Taking ¢= 0 and t= 7 gives the required answers. 


6 AN=* a ear 
TT (2n- 


n=1 


Taking ¢ = 0 gives the required series. 


7 f(t)=34 4xy es eien se 
2 (2n-1) 


n=1 


Replacing ¢ by t - 4m gives the following sine series 


of odd harmonics: 


1 3 42 (=1)" in(2n - 1)t 
iG 1n)- 3.4 sin id 
2/2 we (2Qn-1) 


n 


co n+l 
8 f= y CB sin BE 
Tt n I 


n=l 


2Kwr 1... nant 
9 fHD=4Y = sin 
f(t) 7 sin i 


n=1 


3,6 1 sin(2n - 1)nt 
ime ya ee 
AO=5 nu Gnd 5 
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cos2not 
An’ -1 


A 1_s. — 
11 v(t) ==| 1+=msin at - 2 
d| : bp 


n=1 


200 n 
12 fair sty Cy cos ME! 
3 


n 


n=1 


B ao=§l1 25 ain 28) 


n=l 


15 f(t)= -8 y = re cos(2n - 1)nt 


16 (a) fi)=2- 
3 


lel, 
(b) f(t)= . BS Fs sin 2ntt 


n=1 


4 
io 2) rod ot 


x sin(2n — 1)nt 


co n+l 
(c) fy=2+4 yA cos nat 
3 oy On 


13 
17 f(t)=-m - — cos 2nt 
rea 


8X 1 F 
US eed ree ae 


n=1 


19 fey (-1)"" sain 2(2n- 1)nx 
(2n- 1) 1 


n=1 


n+l 
20 Ao =5sin +2 Sy TT sin ant 
4n°-1 


n=1 


at faie-4A4 


22 T(x y=8KE y 1, ,(2n= lynx 
(n= 15° L 


sin 2ntt 


rag: 
23 f(t)= = cos mt +4 > 
"5 An’ - 1 


n=1 
— 1 
irs 


Ni 


aity 


i sin(2n - 1)nt 


ANSWERS TO EXERCISES 1011 


ntl 
26 (c) 1445 SP sin nt 


n=1 


1 2 — 2 n 
29 (a) -m + § S(-1) cos nt 
6 Le 


ie ae Zens 
TU n 


ar ") 
n=—| COS NT - cos =nT 

nt 2 

2 
+ 2( 38 sin lin == cos lan 
4n 2 8n 2 
+ 3 cos lin = a sin tur) . 
n 2 Tn 2 


£ (32° 16} sint + 4(32 + x? - 6x) sin 2t 
TAZ 8 


3\9 
oe => cos(2n - Int, 2 > sin(2n - l)t 
Er _ iy Téa (2n-1) 


1 
d) -+ 7G 2(2n - 1)nt 
OC ae Z ——— \r 


n=l 


30 e(t)= ie | sin(2n - 1)100n¢ 
T 7 2n- 1 


i(t) = 0.008 cos(100nf — 1.96) 
+ 0.005 cos(300nt — 0.33) 


31 f(t) -“ ys so sin(2n - 11 


n=l 
x,(¢) = 0.14 sin(at — 0.1) + 0.379 sin(3mt — 2.415) 
+ 0.017 sin(Smt — 2.83) 


100 — (-1)"" |. 
32 f(t)=— 2mnt 
S(t) : 2 "Sin 2am 


x(t) = 0.044 sin(2mt — 3.13) — 0.0052 sin(4mr — 3.14) 
33 ¢(1)= +22 + 50 sin some 200 yy cos! 00nnt 
T nm “a 4n-1 
it) = 0.78 cos(S5Omt + (—0.17)) 
— 0.01 sin(100zt + (—0.48)) 
1 = ji n jnmt/2 
35 f(t)==+ -)'-1 
foe; »2) Sal )"- Me 
n#0 


36 (a) ant y Lis. 


n=-09 


tte vfs jnt 


n#0. 
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b) Ssi = 
ee b3 


n=—co 


[(- 1)" ag 1] ene 
cae -1) 


3. — mie _¢_1)" jnt 
OF. ve 


n=—00 


n#0 
(d) 2, 1 ; ein 
T ,. 1-4n 
38 (b) (i) 17.74, (ii) 17.95 
(c) 18.14; (i) 2.20%, (ii) 1.05% 


39 (a) ty = 15,.c, =U 2 guy 
jn 


15, Ou j yo Pu +5).0, 81 -j) 
(b) 15 W, 24.30 W, 12.16 Pr 2.70 W, 0.97 W 

(c) 60W 

(d) 91.9% 


40 0.19, 0.10, 0.0675 


41 (c) (@=0, c, ae: 0, c;=—-2 


1 1 3 
42 (C) Co=7, 1 =5, 2=7%,6;=0 


46 (b) cy =0, c, = \(2m), c) = 0, MSE =0 


7.9 Review exercises 


1 f= ox +y 2c 1)" cos nt 


n=l 


a satan 
Qn-1 n(2n-1) 


- ¥ = sin 2nt 
= 2n 


Taking 7 = 7 gives the required sum. 


2 fit)= aps eos! smn s+ W"1 cost 


n=1 


T 


3 (a) iD Vere ta ieee 


RovT 


(b) -17; 
(c) Taking ¢ = iT gives S = Ly? 


5 f(t)= ae 1)" aa lt 


n=l 


8 f(x)== =o. ame 5 sin(2n - 1)x 


n=l 


fay=4n- mee cos 2(2n =x 
4 (2n- 1) 


n=1 
10 (a) f(t)= 3 2 sin nt 
n=1 


1 
b == 
Oe Paes >Y 


n=l 


5 cos(2n — 1)t 
= =1y 


5 cos(2n - 1)t 


i. Ac 
13 == pep 
(a) f(t) 


T Hl 


5 ee 
Oh 2o=— > 5 


n=l 


(Qn- 


i sin(2n - 1)t 


15 (a) vin=* 0 4 Ssin 2% ep pee aay 
T “4n’-1 T 
(b) 2.5 W, aie 
16 (b) an=ty 1 sin(2n - 1)t 
TT 2n-1 


n=l 


SD =14+ 8 
18 (b) Sin Ot @ cos wt Dames sin Ot - & cos at 
l+@ Ty (2n-1)(1+@) 
a= (4n — 2)n/T 
19 (c) M=1, T =h, T= 20 - 1, Ty = 40 - 34 
(d) £7; -27,4+ 37, -37,+2T, - BT, 


3 _ 67,91, _ 59 1 
(e) zt St + it 3° 


i 
wo t= 1 


CHAPTER 8 


Exercises 


1 24 . 
at+@ 
2 AT*josine St 
3 AT sine’ 


4 8Ksinc 2@, 2K sinc @, 2K(4 sinc 2@ — sinc @) 


5 4sinc @— 4 sinc2@ 


ee ee 

(a+jo) + @ 
Whee shes 
KG x +a 


1 


Fae RETOUR 
(1-@)+3jo 
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13 4sinc2@-2 sinc @ 
14 ST [sinc 5(@ - w)T+sinc S(@+ o)T | 
15 tT [elo sine 4 @- Wy) T 


-j@,)T/2 
+e 70 


sinc $(@+ @)T] 

16 j[sinc(@+ 2) — sinc(@— 2)] 

18 4AT cos wtsinc oT 

19 High-pass filter 

20 nel”! 

21 T[sinc(@—- @,))T + sinc(@+ @,)T] 
QM 


2 2 
Oo- @ 


26 4 nj[5(w+ a) - (@- @)] — 
2 


28 {2, 0, 2, 0} 
29 {2, 0,2, 0} 


32 D(z) = 0.06366 — 0.10660z~ + 0.318312 4 + 0.5z° 
+ 0.318312 — 0.010660z* + 0.06366z "° 


33 D(z) = 0.00509(1 + 2'°) — 0.04221(z* + 2°) 
+ 0.29035(z4 + 2°) + 0.5z° 


8.10 Review exercises 


1 Sin@_ cos2@ 
{7 


(0) @ 


7 a) Ce" - e” H(t) 
a-b 
(b) (i) te*H(¢) (ii) (t- 1 +H) 
8 (a) —sin wo(t+ 4m) (b) cos Wot 
(c) je!" (d) -je"*" 
7 a+2ts 
- a+4ns° 


(b) — (sin 2nsT - cos2msT + 1) 
2s 


CHAPTER 9 


Exercises 
1la@=bec 
2 a=t¢ 


5 For a=0:V=A+ Bx 


10 


12 


15 


16 


19 


20 


21 


22 


23 


25 


ANSWERS TO EXERCISES 


For a > 0: 

V =Asinhat + Bcoshat, where a’ = a/K 
or Ce“ +De“ 

For a < 0: 

V=Acos bt + Bsin bt, where b* = —a/K 


n=-3,2 
a=-3 
(a) Te = (Lon 


(b) v= (rg)v + (re)u, and (rc)W, = W,, 
(c) w,, = (Le)w, 


2(z) = (1 + 2z)/(1 +2)" 
u = sin x cos ct 


ly: ‘ : ; 
u= _ (sin xX sin ct + i sin 2x sin 2ct) 


u= 2 sin aaa) oe ssi 3n(x = ct) a et 
Tt 


25 I l 


ee sin Sx = ct +.. | + i n(x + cl) 


1. 3mx+ct 1 . 5atx+ct 
——— + — sin +. 
I 25 I 


exp{—(x + ct)"}] 


u= x [exp{—(x — ct)*} 
u=}F(x—ct) + + F(x + ct), where 


l-z (0<z<€<1) 
l+z (-l<z=<0) 
= 1 


0 (lz| = 1) 


F(z) = 


x + (-3 — \6)y = constant 
and 
x + (-3 + \6)y = constant 


u =} [4(x + 28)? + («- 31)? - 5] 


1013 


‘S 


x f(x) u(x,0) — u(x, 0.5) u(x, 1) u(x, 1.5) u(x, 2) 
-3.0 0.024893 0 0.025943 0.058509 0.106010 0.180570 
2.5 0.041042 0 0.042774 0.096466 0.174781 0.297710 
2.0 0.067667 0 0.070522 0.159046 0.288166 0.490 842 
-15 0.111565 0 0.116272 0.262222 0.475106 0.681 635 
-1.0 0.183939 0 0.191700 0.432332 0.655692 0.791 166 
0.5 0.303265 0 0.316060 0.585169 0.748392 0.847392 
0 0.5 0 0.393469 0.632120 0.776869 0.864 664 
0.5 0.696734 0 0.316060 0.585169 0.748392 0.847392 
1.0 0.816060 0 0.191700 0.432332 0.655692 0.791 166 
1.5 0.888434 0 0.116272 0.262222 0.475106 0.681635 
2.0 0.932332 0 0.070522 0.159046 0.288166 0.490 842 
2.5 0.958957 0 0.042774 0.096466 0.174781 0.297710 
3.0 0.975106 0 0.025943 0.058509 0.106010 0.180570 
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26 u= sy ee sin(2n - 1)x cos(2n - 1)ct 
T (2n- 1) 


28 Explicit with A = 0.5 


29 


30 


x t=0 ¢=0.25 ¢t=0.5 t=1 t=1.5 

0 0 0 0 0 0 

0.25 | 0 0.0625 0.125 0.179687 0.210937 
0.50 | 0 0.125 0.21875 0.265625 0.269531 
0.75 | 0 0.0625 0.125 0.179687 0.210937 
1.00 | 0 0 0 0 0 
Implicit with A = 0.5 

bi t=0 t= 0.25 t=0.5 t=1 

0 0 0 0 0 

0.25 0 0.0625 0.122 45 0.17407 
0.50 0 0.125 0.224 49 0.2815 
0.75 0 0.0625 0.122 45 0.17407 
1.00 0 0 0 0 
Explicit 

x t=0 +¢t=0.02 t= 0.04 t= 0.06 t= 0.08 
0 0 0.031410 0.062790 0.094108 0.125333 
0.2 | 0 0 0.000314 0.001249 0.003 101 
0.4 | 0 0 0 0.000003 0.000018 
0.6 | 0 0 0 0 0.000 000 
0.8 | 0 0 0 0 0 

1.0 | 0 0 0 0 0 
Explicit 

x t=0 t=0.2 t= 0.4 t= 0.6 
0 0 0 0 0 

0.2 0.16 0.19 0.2725 0.388 75 
0.4 0.24 0.27 0.36 0.508 125 
0.6 0.24 0.27 0.36 0.508 125 
0.8 0.16 0.19 0.2725 0.388 75 
1.0 0 0 0 0 


x t=0 t= 0.2 t= 0.4 t= 0.6 
0 0 0 0 0 

0.2 0.16 0.19 0.2319 0.2785 
0.4 0.24 0.27 0.3191 0.3849 


31 


32 


33 
34 


35 
36 


37 


39 


41 


42 


Explicit 

x t=0 t=0.2 t=0.4 t=0.6 
0 0 0.03 0.12 0.27 
0.2 0.16 0.19 0.28 0.43 
0.4 0.24 0.27 0.36 0.51 
0.6 0.24 0.27 0.36 0.51 
0.8 0.16 0.19 0.28 0.43 
1.0 0 0.03 0.12 0.27 


Implicit (symmetric as in the explicit case) 


x t=0 t=0.2 t=0.4 t=0.6 


0 1 0.03 0.08 0.1495 
0.2 0.16 0.19 0.24 0.3099 
0.4 0.24 0.27 0.32 0.39 


— 1 9 2 3 
u = 5 alexp(—; Km) cos; mx 


+ exp(—} 72) cos } mx] 


Ay = 2/1N 
a=-1,x=-] 
B=2, u = —we“sin(x — 21) 


The term represents heat loss at a rate proportional to 
the excess temperature over 6. 


= —K(n+4) n't 
u=S'a, exp ss cos (n+>)se 
n=0 l 2 l 
where 
Pe eee 2(=1) | 


n 0 22 
Qn+1)n (Qn+1)n 


u(0, t) = u(/, t) = 0 for all ¢ 
u(x, 0) = 10 for0<x</ 


x t=0 ¢=002 ¢=004 ¢=0.06 ¢=0.08 ¢=0.1 


0 0 0 0 0 0 0 

0.2 | 0.04 0.08 0.1 0.12 0.135 0.1475 
04 | 0.16 0.2 0.24 0.27 0.295 0.315 
0.6 | 036 0.4 0.44 0.47 0.495 0.515 
0.8 | 0.64 0.68 0.7 0.72 0.735 0.7475 
1.0 1 1 1 1 1 1 


At t= 1 with A= 0.4 and At = 0.05 
Explicit 


0 0.2 0.4 0.6 0.8 1.0 
0 0.1094 0.2104 0.2939 0.3497 0.3679 


u 
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43 


44 


46 


47 


48 


50 


51 


54 


55 


56 


57 


Implicit 
x |0 0.2 0.4 0.6 0.8 1.0 
u|}0 0.1082 0.2095 0.2954 0.3551 0.3679 
x t=0 ¢=002 ¢=0.04 t large 
0 0 —0.04 -0.0799 > -l 
0.2 0.16 0.12 0.0803 > -0.8 
0.4 0.24 0.2002 0.1613 7 -0.6 
0.6 0.24 0.2012 0.1657 7 -04 
0.8 0.16 0.1269 0.1034 74 -0.2 
1 0 0 0 > 0 
u =e” sinnx - ce °™’sin 3nx+4e°” sin Sax 
sete 
+ 

g=x°y + sin x iain 

r ry 
u(r, @) = 3-sinO — (2) sin(30) 

a a 


v = const gives circles with centre (=, 0) and 
radius 1/|v —1| B 

u = const gives circles with centre (1. =) and 
radius 1/|z| - 


“Sin nTx 
u=xt+s 2 y Saat 
nsinh nt 


n=l 


x {sinh nay + (-1)" sinhnn(1 — y)} 


Boundary conditions are u(0, y) = u(a, y) = 0 
0<y <a; and u(x, 0)=0,0 Sx Sa, ux, a) =u, 
O<x<a 
5 
9) a saomll SI 
3 
an a ee | ee 


For Ax = Ay = 0.5 
For Ax = Ay = 0.25 


u(0.5, 0.5) = 0.3125 
u(0.5, 0.5) = 0.3047 


At two sample pone 

For Ax = Ay= 5, u(0.5, 0.5) = 0.6429 and 
u(0.5, 1) =0. S714 

For Ax = Ay = }, u(0.5, 0.5) = 0.6379 and 


u(0.5, 1) = 0. 5602 


u(1, 1) = 10.674, u(2, 1) = 12.360, u(3, 1) = 8.090, 
u(1, 2) = 10.337, u(2, 2) = 10.674 


58 


59 


60 


61 


62 


63 


64 = 


66 


68 


69 


71 
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h = 1/2 gives (0.5, 0.5) = 1.8611 and 
(0.5, 1) = 1.3194 


For / = 1/4 dis given in the table 

y 

ll 2 1.601 5666 1.286 7647 1.056 5216 1 
0.75 | 2.4375 1.967955 1 1.581 8015 1.257 2287 1 
0.5 2.75 2.266 5772 1.846 5073 1.437 4669 1 
0.25 | 2.9375 2.5174715 2.131 4338 1.693 006 4 1 
0.0 3 2.75 25 2725 1 
% 0 0.25 0.5 0.75 1 


(0, 0) = 1.5909, (0, !) = 2.0909, (0, 2) = 4.7727, 
(4, 0) = 1.0909, o(2, 0) = 0.7727 and other values 
can be obtained by symmetry. 


(a) uw, = 1/35, uw = 6/35 
(b) wu, =0.1024, u, = 0.0208, uw; = 0.2920, u, = 0.2920, 
us = 0.0208 


Has the same solution as Exercise 57. 


u(0, 0) = 1.6818, u(0, 5 1) = 2.2485, u(0, 2)y= 5.3121, 
u(t, 0) = 1.1152, u(2, 0) = 0.7727 and other values by 
symmetry. Compare with Exercise 59. 


“(ta an (Se tan5) + tan" SZ cot 5] 
T a 2 a-r 2 


T(r, 0) = 


z-L } 
(x +a) +y ‘| 


Parabolic; r=x—yands=x+y gives u,, = 0 
Elliptic; r=—3x + y and s=x + y gives 

8(u,, + U,,.) — 9u, + 3u, +u=0 

Hyperbolic; r=9x + y and s=x+y gives 49u,,.— u,,=0 


u=f(2x + y) + g(x - 3y) 


(a) elliptic; 
(b) parabolic; 
(c) hyperbolic 


For y < 0 characteristics are (—y)*” + 3.x = constant 
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72 elliptic if|y| << 1 
parabolic ifx =0 ory=+1 
hyperbolic if |y| > 1 


73 p> qorp <~—q then hyperbolic; p = q then parabolic; 
—q <p <q then elliptic 


9.10 Review exercises 


3 y=4e""" sin( 317 }( cos @3tt+ | sin ws!) 
I 2@3T 


2 1/2 
where @, = 3n{(1 - — Ps | 
l 360 CT 


5 As; = 80,l/m2(2n + 1) 


6 T=Ty + Gfl — erf(x/2\(KA))] 


7 Explicit 
x t=0 t= 0.004 t= 0.008 
0 1.0000 0.9600 0.9296 
0.2 1.0000 1.0000 0.9960 
0.4 1.0000 1.0000 1.0000 
0.6 1.0000 1.0000 1.0000 
0.8 1.0000 1.0000 0.9960 
1.0 1.0000 0.9600 0.9296 
Implicit 
x t=0 t= 0.004 t= 0.008 
0 1.0000 0.9641 0.9354 
0.2 1.0000 0.9984 0.9941 
0.4 1.0000 0.9999 0.9996 
0.6 1.0000 0.9999 0.9996 
0.8 1.0000 0.9984 0.9941 
1.0 1.0000 0.9641 0.9354 
9 
y=l 1 0.928 5925 
y=0.5 0.9875743 0.9569621 0.9379995 
y= 1 0.9849808  0.9647746 0.960 1934 
x=0 x=0.5 x=1 x= 1,5 
11 k=-3 
12 z=x—y, valid in the region x = y 
2 n+l 
14 Aaya = ae 
m(2n+ 1) cosh [oes tnt bee 
a 


15 u(x, th= 7y" . sin nx cos nit 
Ton 


n=1 
17 @=Acos(px)e“cos wt, where @ = c’p” — 1K? 


18 Onr=a, v,=0, so there is no flow through 
the cylinder r= a. As r > ~, v, ~ Ucos 0 and 
Vv, — —Usin 8, so the flow is steady at infinity and 
parallel to the x axis. 


CHAPTER 10 


Exercises 


1x=l,y=1,f=9 


3 Original problem: 
20 of type 1, 50 of type 2, profit = £1080, 70m 
chipboard remain 


Revised problem: 
5 of type 1, 75 of type 2, profit = £1020, 5 m chipboard 
remain 


4 4kg nails, 2 kg screws, profit 14 p 
5 9 of CYL1, 6 of CYL2 and profit £54 


6 LP solution gives x, = 66.67, x, = 50, f= £3166.67. 
Profit is improved if more cloth is bought, up to a 
maximum when the amount of cloth is increased to 
600 m then x, = 0, x, = 150 and f= £4500. 

7 Fork = 60: x, = Bx), =0,2= 2k 
For 60 = k= 10: x, = 6, x, =7, z= 140 + 6k 
For k < 10: x, = 0, x, = 10, z = 200 

8 x, =1, x, = 0.5, x5 = 1, x, =0, f= 6.5 


9 B1, 0; B2, 15.000; B3, 30 000; profit £21 000 


10 Long range 15, medium range 0, short range 0, 
estimated profit £6 million 


11 Many solutions of the form x, = 1.5 — 1.54, x, = 0, 
x; = 2.5 — 1.5t, x, =3t where 0 St S 1 giving f= 14. 


12 x=l,y=4,f=9 
13 x,=1,x, = 10, f=20 
14 Boots 50, shoes 150, profit £1150 


15 B1, 0; B2, 10000; B3, 40 000; profit is down to 
£20000 


16 x=3,y=0,z=?,f=# 
17 x, =2, %, =0,%,=2,%,=0,f= 12 


18 36.63% of A, 44.55% of B, 18.81% of C, profit per 
100 litres £134.26 
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19 6 of style 1, 11 of style 2, 6 of style 3, total profit 
£37 500 
20 x, = 2500 m’, x, = 1500 m’, x; = 1000 m’, profit £9500 
21x= oy = t 
22 x=taandy=0 
23 x =al/2, y= b/\2, area = 2ab 


24 Several possible optima: (0, 3, 0); (3, 3, $); 
(6 — 3¢, 0, A) for any ¢ 


25 (0, 1, 1); (0, -1, -D; 2, -1, 1/J7; -@, -1, D//7 


26 For given surface area S, b = c = 2a, where a’ = 5 S 
and V = 4a’ 


27 A=—1.83, B = 0.609, = 81.4 


28 For a= 0 minimum at (0, 0); for « < 0 minimum at 
(2a, —300)/5 


29 (a) Bracket (without using derivatives) 0.7 <x < 3.1 
(b) Iteration 1: 


a 0.7 fia) 2.7408 


b 1.9 f(b) 2.177 


c 3.1 fic) 3.2041 


Iteration 2: 


a 0.7 fla) 2.7408 


b 1.7253 fb) 2.0612 


c 1.9 fc) 2197 


gives b = 1.5127 and f(b) = 1.9497 


(c) 


Iteration | Iteration 2 


x 0.7 3.1 0.7 1.5129 
F(x) 2.7408 3.2041 2.7408 1.9498 
f(x) 4.8309 0.9329 -4.8309 0.4224 
gives x = 1.1684 and f= 1.9009 


30 (a) Iteration 1: 


a 0 f(a) 0 
b 1 i) 0.420 74 
c 3 iG) 0.014 11 


ANSWERS TO EXERCISES 1017 


Iteration 2: 


a 0 f(a) 0 
b 1 f(b) 0.420 74 
c 1.5113 fc) 0.303 96 


gives x = 0.989 79 and f= 0.422 24 


(b) ; : 
Iteration | Iteration 2 
x 0 1 0 0.8667 
F(x) 0 0.4207 0 0.4352 
f(x) 1 —0.1506 1 —0.0612 


gives x = 0.8242, f= 0.4371, f’ =-0.0247 
31 (a) Iteration 1: 


a 1 fla) 0.232 54 
b 1.6667 f(b) 0.255 33 
Cc 3 fc) 0.141 93 
Iteration 2: 
a 1 f(a) 0.232 54 
b 1.6200 f(b) 0.257 15 
c 1.6667 fic) 0.255 33 
gives x = 1.4784 and f= 0.260 22 

(b) Iteration 1: 
x 1 3 
F() 0.232 54 0.141 93 
vies) 0.135 34 —0.087 18 
Iteration 2: 
x 1 1.5077 
F() 0.232 54 0.259 90 
T(x) 0.135 34 —0.013 68 


gives x = 1.4462, f= 0.260 35, f’ = —0.000 14 
(c) Convergence in 6 and 3 iterations to x = 1.446, 
f= 0.2603 


32 x= 1, Amp = 2; X= 0, Amp, = 1.414 21; x =-1, 
A wax = 1-732 05. One application of the quadratic 
algorithm gives x = —0.148 26 and Ainax = 1.3854. 


max 


34 (a) After five iterations x = 2.0492 and f= 1.8191. 
(b) After five iterations x = 2.1738, f= 0.0267059. 
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35 


36 


37 


38 


39 
40 


Iteration 1: 


civena 
1 1 


a= -1 : =-t, Vf= l 
-1 -1| 


Iteration 3: 


1 
Vf= |" 
5 


1 
a=| °*|, f=-0.9, 
3 
5 


Steepest descent gives the point (—0.382, —0.255) and 
f= -0.828 


f -29.0000 -1.5023 -0.4523 -0.0764 -0.0248 -0.0165 +0 

x 2.0000 1.1523 0.5022 0.6214 :0.4948-—(0.5185 3 0.5 
y 2.0000 2.1695 1.8214 1.7539 1.6654 1.6394 9 1.5 
z 2.0000 0.4741 0.7943 0.9630 -:1.0170 1.0301 > 1 

f 29.0000 1.2448 0.1056 0.0026 0.0000 0 

x 2.0000 0.2727 0.4245 0.4873 0.4995 —>0.5 
y 2.0000 1.7273 1.5755 1.5127 1.5005 1.5 
Zz 2.0000 1.4545 1.1510 1.0253 1.0009 +1 

y =0.2294x and y = 0.5x — 0.2706, cost = 5.974 


(a) After step 1 


(ph me ofa 
2 1.6 


H= 0.1385 0.1923 
0.1923 0.9615 
After step 2 the exact solution x = 1, y= 1 is obtained 
(b) After cycle 1 


| 0.5852 ~0.3918 
a,=|0 » f= 1.0662, 8, = |-1.7557 
| 0.2926 0.7822 
| 0.3681 0.1593 0.4047 
H,=/ 0.1593 0.9632 0.1002 
|-0.4047 0.1002 0.7418 
After cycle 2 
1.0190 
a,=!| 0.9813|, f= 2.999 x 10°°, 
—0.0372 


0.0046 
82 =|—0.0012 
0.0027 


41 (a) After cycle 1 


i= seer f=02404, g= a 
—0.061 —0.242 
ec 0.995 aa 
—0.062 0.258 
After cycle 2 the minimum at x = 0, y = 0 is 
obtained 
(b) After cycle 1 
—0.0732 0.0386 
a,=}| 0.8344], f=90.1563, g, = |0.1564 
0.4522 0.6296 
0.4425 0.3669 0.0998 
H, = | 0.3669 0.7585 —0.0657 
0.0198 —0.0657 0.9821 
After cycle 2 
—0.1628 —0.2006 
a=] 0.7747|, f= 0.0321, g2= |-0.1207 
0.0525 0.0630 
0.2820 0.1819 —0.0498 
H,=] 0.1819 0.5452 —0.2380 
—0.0498 -0.2380 0.8429 
The method converges tox =0, y=1,z=0. 
43 (a) (0,0) 
(b) 
f 1.3125 0.0764 0.0072 0.0007 0.0004 0.0000 >0 


x 0.5000 —0.0950 0.0057 —0.0251 —0.0079 —0.0032 — 0 
0.9742 0.9978 > 1 
1.0014 > 1 


y 0.5000 0.9165 0.9276 0.9633 


z 0.5000 0.7380 0.9674 1.0009 1.0044 


10.7 Review exercises 


1 x, = 250, x, = 100, F = 3800 

2 x, =22, x, =0, x; = 6, profit £370 

3 Standard 20, super 10, deluxe 40, profit £21 000 

4 2kg bread and 0.5 kg cheese, cost 210 p 

5 Maximum at (1, 1) and (-1, —1), with distance = \2 
Minimum at (3, —\) and (—\, \4), with 


(2 


distance = |; 
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6 Sides are 3+ and 24+ 


Vi0 V10 


7 (4,0, $), with distance 2.683 


8 (1, 2, 3) with F = 14, and (-1, -2, -3) with F =-14. 
(1, 2, 3) gives the global maximum and (0, 0, 0) gives 
the global minimum. 

9 (i)b=c (ti)a=b=c 


10 1? = 3n’/b, r? = 3a’/2b 


11 k=2.19 

12 Bracket: 
R 3.5 3.5 9.5 
Cost 1124 704 1418 


Quadratic algorithm gives R = 6.121 and cost = 802, 
so R=5.5 still gives the best result. After many 
iterations R = 4.4 and cost = 579 


13 Quadratic algorithm always gives x = 0.5 for any 
intermediate value. However, 


0.7524 
0.5629 
0.6051 
1.0000 


0.7508 
0.6051 
0.6243 
1.0000 


0.7729 — 0.7500 
0.3147 
0.5000 


1.0000 


0.7584 
0.5000 
0.5629 
1.0000 


=> 0.6514 


aera SS 


14 Maximum at @= 5.01 rad, minimum at 8 = 1.28 rad 


15 44 mph 


16 At iteration 2 
(a) x =—0.0916, y = —0.1375, f= 0.0326 
(b) x =—0.1023, y = —0.1534, f= 0.0323 
(c) x =—0.1007, y = 0.1519, f= 0.0323. The exact 
minimum is at x = —0.1026, vy = —0.1540, f= 0.0323 


17 Maximum of 1.056 at X= 0, Y= 0.4736, minimum of 
0.5278 at X¥ = +0.25, Y=2 


18 Partan 


x3= 


X4= 


19 Start valu 


20 (a) ay = 


a, = 


(b) a) = 


a,= 


a= 
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_ [0.333 
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05] r-925 

| 0.5 

BPS) goa i935 
| 0.5 

es 


7 h f= 1 (no improvement) 
1 


3 
2 


| 0 


, F=0.0625 
0 


, F=0.0039 
025 
[0.375 


, F=0.0002 
| -0.375 


1 


} F = 0.3125 
1 


, F= 0.0664 
| 3.667 


1019 


| , f=-5 (ready for next iteration) 
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24 Bracket gives 


a [F(a - 3° 
14 0.0776 
1.5 0.0029 
1.6 0.0369 


Quadratic algorithm gives a@* = 1.5218 and f=9 x 10° 


CHAPTER 11 


Exercises 
1 (a) (762, 798) (b) 97 
2 76.1, (65.7, 86.5) 
3 (8.05, 9.40) 
4 (71.2, 75.2), accept 
5 (2.92, 3.92) 
6 (24.9, 27.9) 


7 95% confidence interval (53.9, 58.1), criterion 
satisfactory 


8 (—1900, 7500), reject 


9 90%: (34, 758), 95%: (—45, 837), reject at 10% but 
accept at 5% 


10 90%: (0.052, 0.089), 95%: (0.049, 0.092), reject at 
10% but accept at 5%. Test statistic leads to rejection 
at both 10% and 5% levels, and is more accurate 


11 203, (0.223, 0.327) 


12 90%: (—0.28, —0.08), 95%: (—0.30, —0.06), accept at 
10% but reject at 5% 


13 (0.003, 0.130), carcinogenic 


14 (a) X: (0.34, 0.53, 0.13), Y: (0.25, 0.31, 0.44) 
(b) 0.472, (c) E(X) =1.79, Var(X) = 0.426, 
E(Y) = 2.19, Var(Y) = 0.654, py» =—0.246 


17 (a) 0.552, (b) 0.368 
18 0.934 

19 0.732 

20 (0.45, 0.85) 

21 (0.67, 0.99) 


22 0.444, 90%: (0.08, 0.70), 95%: (0.00, 0.74), just 
significant at 5%, rank correlation 0.401, significant 
at 10% 


23 (a) 6, (b) 0.484, 
(c) fr) = 6(5 —x + 5x°), f(y) = OC — yyy 
24 0.84 
25 a=1.22,b=2.13 
26 a= 6.315, b = 14.64, y = 226 


28 (a) a= 343.7, b= 3.221, y= 537; 
(b) (0.46, 5.98), reject; (c) (459, 615) 


29 a =0.107, b = 1.143, (14.4, 17.8) 
31 120Q 

32 A= 2.66, C=2.69 x 10°, P= 22.9 
33 a= 7533, b=-1.059, y= 17.9 
34 7° = 2.15, accept 

35 x = 12.3, significant at 5% 

36 7° = 1.35, accept Poisson 

37 7° = 12.97, accept Poisson 

39 7° = 1.30, not significant 

40 7 = 20.56, significant at 5% 

41 7 = 20.7, significant at 0.5% 


42 7° = 11.30, significant at 5% but not at 1%, for 
proportion 95%: (0.111, 0.159), 99%: (0.104, 0.166), 
significant at 1% 


43 c=4, M,(t)=4/(t- 2), E(X) = 1, Var(X) =! 
45 0.014 
46 0.995 


48 Warning 9.5, action 13.5, sample 12, UCL = 11.4, 
sample 9 


49 UK sample 28, US sample 25 

50 Action 2.93, sample 12 

51 Action 14.9, sample 19 but repeated warnings 
52 (a) sample 9, (b) sample 9 
53 (a) sample 10, (b) sample 12 
54 sample 10 

55 sample 16 


56 (a) Repeated warnings, 
(c) sample 14 


(b) sample 15, 


58 sample 11 
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59 


60 
61 
63 


65 
66 
67 
68 
69 
70 
71 
72 
73 
74 
75 


Shewhart, sample 26; cusum, sample 13; 
moving-average, sample 11 


0.132 
0.223, 0.042 
(a) +,  (b) 24, — (©) 0.237, 


(d) 45 min, (e) 0.276 

Mean costs per hour: A, £200; B, £130 
6 

Second cash desk 

29.4% 

Sabotage 

P(C|two hits) = 0.526 


1 


2 


(a) 0.0944, (b) 0.81 
(a) 3, (b) + (2"*7" 
AAAA 


1.28:1 in favour of Poisson 
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77 2.8:1 in favour of H, 
78 12.8:1 in favour of H, 


11.12 Review exercises 
1 Z=0.27, accept 

2 (0.202, 0.266) 

3 (96.1 x 10°, 104.9 x 10°) 


4 7° = 3.35 (using class intervals of length 5, with a 
single class for all values greater than 30), accept 
exponential 


5 Outlier 72 significant at 5%, outlier included 
(7.36, 11.48); excluded (7.11, 10.53) 


6 ¥ = 20.0, significant at 2.5% 
7 Operate if p > = 


8 Cost per hour: A, £632.5; B, £603.4 
pa 


Pat pa 


9 (a) P(input 0|output 0) = etc. 


(b) p<a<l-p 
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A 


abscissa of convergence of Laplace 
transforms 354 
AC circuits (application) 335-6 
action limits in control charts 965 
Adams-Bashforth formulae 133 
Adams-Morton formulae 140 
addition of matrices 4 
addition rule 907 
adjoint matrix 5 
algebraic multiplicity of eigenvalues 23 
aliasing error 688 
alternative solutions in linear 
programming 854 
amplified gain 464 
amplified input 462 
amplitude gain 464 
amplitude modulation (application) 
703-9 
demodulation stage 707-8 
final signal recovery 708-9 
information-carrying signal 706—7 
and transmission 705—6 
amplitude ratio 464 
amplitude spectrum 615, 648, 711 
analogue filters 545 
application 700-2 
analytic function 283 
applied probability 906 
arbitrary constant 735 
arbitrary function 735-40 
arbitrary inputs in transfer 
functions 446-9 
Argand diagram 259 
artificial variable 862 
associative law 4 
asymptotically stable system 104 
attenuated input 462 
attribute 964 
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augmented matrix 9 
average power 621 


B 


base set 560 

basic variables 849 

Bayes’ theorem 986-90, 987 

applications 988—90 
derivation 986-8 

beams, bending of 424—7 

bending of beams 424—7 

Bernoulli, 560 

Bernoulli distribution 909 

Bessel’s equality 628 

BFGS method 890 

bilateral Laplace transform 349, 658 

bilinear mappings of complex 
functions 273-9 

bilinear transform method 549 

binding constraints 848 

binomial distribution 909 

Blackman window 717-18 

block diagram algebra 429 

blood-flow model (application) 834-7 

Bode plots 466 

Boole, George 482 

boundary conditions in partial differential 
equations 826-30 

boundary-value problems in differential 
equations 161-2 

bracket 877 

bracketing procedure 876-7 

breakpoint 467 

Brigham, E.E. 693 

Broyden 890 

Butterworth approximation 700 

Butterworth filter 545 


1024 


INDEX 


C 


canonical form of matrices, reduction to 39, 
39-53 
diagonal form 39-42 
Jordan canonical form 42-5 
quadratic forms 47-53 
canonical representation of equations 99 
capacitor microphone (application) 
107-10 
capacitors 382 
carrier signal 655 
Carslaw, H.S. 823 
Cauchy-Goursat theorem 321 
Cauchy-Riemann equations 283-7, 292 
Cauchy’s conditions in partial differentiation 
equations 826, 827 
Cauchy’s integral theorem 325 
Cauchy’s theorem 320-7 
causal functions 349 
causal sequences 483 
Cayley-Hamilton theorem 56 
central difference scheme 140 
central limit theorem 910 
proof 956-7 
chain rule 187 
Chapman, M.J. 545, 549, 702, 709 
characteristic equation 429, 510 
characteristic function 953 
characteristic polynomial 15 
characteristics in partial differential 
equations 745 
chemical processing plant 
(application) 896-8 
Chi-square distribution and test 
contingency tables 949-51 
circular frequency 562 
closed boundary 826 
coefficients of Fourier series at jump 
discontinuities 599-601 
collocation methods 164 
column rank matrix 66, 67 
column vector 3 
columns 761 
commutative law 4 
not satisfied 4 
companion form 83 
complement of event 907 
complementary function in Laplace transform 
methods 373 
complementary function of matrices 90 
complex differentiation 282-94 
Cauchy-Riemann equations 283-7, 292 
conjugate functions 288-90 
harmonic functions 288-90 
mapping 290-4 


946-8 
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complex form of Fourier series 609 
complex frequency 346 
complex frequency domain 348, 648 
complex functions 259-82 

bilinear mappings 273-9 

inversion 268-73 

linear mappings 261-8 

polynomial mappings 280-2 
complex integration 

fundamental theorem of 321 
complex series 295-307 

Laurent series 303-7 

power series 295-9 

Taylor series 299-302 
composite-function rule 187 
conditional distribution 929 
conditional probability 907 
confidence interval for mean 914-17, 

915 

conformal mapping 290 
conjugate functions 288-90 
conjugate-gradient methods 888 
conservative force 218, 246 
contingency tables 949-51 
continuity correction 910 
continuity equation 248 
continuous Fourier spectra 648-50 
continuous Fourier transform 684—92 
continuous random variables 908 
continuous source 821 
continuous-time systems 347 
continuous variables 907 
contour integral 317-20 
contour integration 317-34 

Cauchy’s theorem 320-7 

contour integrals 317-20, 318 

deforming 321 

evaluation of definite real integrals 

331-4 

residue theorem 328-31, 329 
control charts 964 
controllable modes in matrics 100 
convergence 

in Fourier series 584—7 
convergence rate of eigenvalues 32 
convolution 

in discrete-linear systems 524—7 

in Fourier transforms 673—5 
convolution integral 443 
convolution sum 525 
Cooley, J.W. 638, 693 
corner frequency 467 
correlation 929-33, 930 

partial 933 

rank 936-7 

and regression 943 

sample 933-5 


coupled first order equations 151—6 


Courant, Fredricks and Levy (CFL) 
condition 765 

covariance 929-33, 930 

Crank-Nicolson method for solution of 
heat-conduction/diffusion 
equation 782 

cumulative distribution function 908 

curl-free motion 209 

curl of a vector field 206-9 

current 382 

current in field-effect transistor 
(application) 338-40 

customers 975 


Cusum control charts 968-71 


D 


D’Alembert, J. de 560 
D’Alembert solution in partial differential 
equations 742-51, 745 
damped sinusoids 360 
dampers 386 
Danzig 847 
Davidon 888 
DD transform 547-53 
definite real integrals, evaluation of 331-4 
deflation methods in matrices 34 
deforming the contour 321 
degeneracy of a matrix 26 
degree of belief 989 
degrees of freedom 919 
delay theorem 397 
delta function 413 
delta operator 547 
delta operator (application) 547-53 
q (shift) operator 547 
dependent variable 259 
derivatives 
Laplace transforms of 370-1 
of scalar point function 199-202 
of vector point function 203-13 
curl of a vector field 206-9 
divergence of vector field 204-6 
vector operator 210-13 
determinants 
of mappings 274 
of a matrix 3,5 
DFP method 888, 889-90 
diagonal matrix 3, 40, 66 
diagonalization 40 


difference between means 921-2 
difference equation 482 
in discrete-time systems 502-3 


solutions 504-8 
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differential 196 
differential equations 
Laplace transform methods on 370-80 
ordinary differential equations 372-7 
simultaneous differential 
equations 378-80 
step and impulse functions 
transforms of derivatives 
transforms of integrals 
numerical solution of 
boundary-value problems 
coupled first order equations 
first order 117-51 
on engineering problems 
Euler’s method 118—24 
local and global truncation errors 
134-6 
multi-step methods 128-34 
predictor-corrector methods 
136-41 
Runge-Kutta methods 
software libraries on 
stiff equations 147-9 
functional approximation methods 
164-70 
higher order systems, state-space 
representation of 156-9 
method of shooting 162—4 
see also partial differential equations 
differentiation of Fourier series 597-8 
diffusion equation in partial differential 
equations 725, 728-31 
solution of 768-84 
Laplace transform method 772-7 
numerical solution 779-84 
separation method 768-72 
sources and sinks for 820-3 
digital filters (application) 709-15 
and windows 715-18 
digital replacement filters 
Dirac delta function 413 


403-7 
370-1 
371-2 


161-2 
151-6 


125-7 


141-4 
149-51 


546-7 


direct form of state equations 89-91 
directional derivative 201 
directional field 118 

Dirichlet, L. 561 

Dirichlet’s conditions 561, 584 


for Fourier integral 641 
in partial differentiation equations 826, 
828, 829 
discrete Fourier transform 680—4 
discrete Fourier transform (DFT) pair 683 
discrete frequency spectra 615, 639 
discrete-linear systems 509-26 
convolution 524—7 
impulse response 515-18 
stability 518-24 
discrete-time Fourier transform (DTFT) 710 
discrete-time signal 482 
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INDEX 


discrete-time system 347 
constructing 549-51 
design of (application) 544—7 
analogue filters 545 
digital replacement filters 546—7 
difference equations in 502-3 
discrete variables 907 
discretization of continuous-time state-space 
models 538—43 
Euler’s method 538-40 
step-invariant method 540-3, 541 
disjoint events 907 
displacement 386 
dissipative force 218 
distensibility 836 
distribution 414, 907 
of sample average 913-14 
distributive law 4 
domain of dependence 746 
domain of function 259 
domain of influence 746 
dominant eigenvalue 31 
double integrals 219-24, 220 
duality property 656 
Duhamel integral 443 
dynamic equations 84 


E 


echelon form of a matrix 9 
eigenvalues 2, 14-30, 17 
characteristic equation 15-17 
method of Faddeev 16 
and eigenvectors 17-22 
pole location 470-1 
and poles 470 
repeated 23-7 
useful properties 27-9 
eigenvectors 2, 14, 17 
electrical circuits (application) 382-6 
electrical fuse, heating of (application) 
174-8 
element of a matrix 3 
elliptic equations 824 
energy 663 
energy signals 668 
energy spectral density 664 
energy spectrum 664 
engine performance data (application) 
958-64 
mean running times and 
temperatures 959-62 
normality test 962-3 
equal matrices 3 
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equality-constrained optimization 844 

equality constraints in Lagrange 
multipliers 870—4, 871 

equivalent linear systems 99 

essential singularity 308 

Euler, L. 560 

Euler’s formula 563 

Euler’s method 538-40 

Euler’s method on differential 
equations 118—24, 120 

analysis 122—4 

even functions in Fourier series 
expansion 573-7 

even periodic extension 589 

events 907 

Everitt, B.S. 951 

exact differential 197 

excitation term 372 

expected value 907 

explicit formula 

for solution of heat-conduction/diffusion 

equation 779 

explicit methods in partial differential 
equations 765 

exponential distribution with parameter 976 

exponential form of Fourier series 609 

exponential modulation theorem 358 

exponential order of functions 353, 354 


F 


Faddeev method on eigenvalues 16 

faltung integral 443 

Fannin, D.R. 717 

feasible basic solution 849 

feasible region 847 

Fermat, Pierre de 871 

Feshbach, H. 830 

Fick’s law 729 

field-effect transistor (application) 338—40 

filter length 713 

filters 545 

final-value theorem 439-40 

finite calculus 482 

finite difference methods 482 

finite-difference representation 762 

finite-difference techniques 164 

finite-element method 164 

finite elements in partial differential 
equations 802-14 

finite impulse response (FIR) 715 

first harmonic 562 

first order method on differential 
equations 123 


first shift property of z transforms 490-1 
first shift theorem in inverse Laplace 
transform 358, 367-9 
fixed point 261 
Fletcher 888 
Fletcher, R. 890, 891 
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folding integral 443 
forcing term 372 
Forsyth, R. 990 
Forsythe, W. 548 
Fourier, Joseph 560 
Fourier coefficients 563 
Fourier cosine integral 642 
Fourier integral representation 640 
Fourier series 560 
coefficients at jump discontinuities 
599-601 
complex forms 608—23 
complex representation 608-12 
discrete frequency spectra 615-21 
multiplication theorem 612-13 
Parseval's theorem 612, 614 
power spectrum 621-3, 622 
differentiation of 597-8 


functions defined over finite interval 587-94 
full-range series 587-9 
half-range cosine and sine series 589-93 


integration of 595-7 
orthogonal functions 624-9 
convergence of generalized series 
627-9 
definitions 624—6 
generalized series 626—7 
Fourier series expansion 561-87, 563 
convergence 584—7 
even and odd functions 573-7 
Fourier’s theorem 562-6 
functions of period 2B 566-73 
functions of period T 580-3 
linearity property 577-9 


periodic functions 561-2 
Fourier sine integral 642 
Fourier transforms 346, 638—50 


continuous Fourier spectra 648-50 

in discrete time 676—99 
continuous transform 684—92 
fast Fourier transform 693-9 
sequences 676—80 

Fourier integral 638-43 

Fourier transform pair 644-8 

frequency response 658—62 

and Laplace transform 658-60 

properties of 652-7 
frequency-shift 654—5 
linearity 652 
symmetry 655-7, 656 
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time-shift 653-4 


step and impulse functions 663-75 
convolution 673-5 
energy and power 663-72 


Fourier’s theorem 562-6 

Fredricks 765 

frequency 562 

frequency components in Fourier series 
615 

frequency domain 348 

frequency-domain filtering 
(application) 703-9 

frequency-domain portrait 648 

frequency response 

in Fourier series 603-6 
in Fourier transform 658—62 

frequency response (applications) 462-9, 
464 

frequency response plot 469 

frequency-shift property 654—5 

frequency spectrum 615 

frequency transfer function 661, 679 

Fryba, L. 831 

full-rank matrix 66, 67, 77 

functional approximation methods in 


differential equations 164—70 
functions 259 
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with localized support 168 
of periiod 2B 566-73 
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fundamental mode 562 
fundamental theorem of complex 
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Gauss’s divergence theorem 241—4 

generalized calculus 414 

generalized derivatives 420 

generalized form of Parseval’s theorem 
628 


generalized Fourier coefficients 627 
generalized Fourier series 627 
generalized Fourier transforms 666 


generalized functions 413 
generating function 484 
geometric distribution 979 
geometric moving-average (GMA) 

charts 971 
Gerschgorin circles on matrices 
Gibbs, J.W. 586 
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Goldfarb 890 
Goodall, D.P. 545, 549, 702, 709 
Goodall, R.M. 548 
goodness-of-fit tests 946-51 
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Goodwin, G.C. 548, 553 
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Green’s functions 792, 819 


Green’s theorem 224-9, 226, 320, 829 
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Haberman, R. 725, 819 

half-range cosine series expansion 591 

half-range Fourier series expansion 591 

half-range sine series expansion 591 

Hamming window 717-18 

Hanning window 717-18 

harmonic components in Fourier series 615 

harmonic functions 288-90 

harmonic functions (application) 336—40 

heat-conduction in partial differential 

equations 725, 728-31 
solution of 768-84 

Laplace transform method 772-7 
numerical solution 779-84 
separation method 768—72 
sources and sinks for 820-3 

heat transfer (application) 250—4 

using harmonic functions 336—8 

heating fin (application) 898—900 

heaviside step function 392-5 

heaviside theorem 397 

Helmholtz equation 734 

Hessian matrix 884 

higher order systems, state-space 

representation of 156-9 
hill climbing 867, 875—95 


advanced multivariable searches 888-91 
least squares 892-5 
single multivariable searches 882-7 


single-variable search 875-81 
holomorphic function 283 
Hooke’s law 386 
L’H6pital’s rule 313 
Householder methods 35 
Hunter, S.C. 812 
Hush, D.R. 717 
hyperbolic equations 825 
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Ifeachor, E.C. 717 
image set 259 
implicit formula for solution of 
heat-conduction/diffusion 
equation 779 
implicit methods in partial differential 
equations 765 
improper integral 348 
impulse forces 413 
impulse functions 413-14, 415-18 
in Fourier transforms 663-75 
Laplace transforms on 403-7 
impulse invariant technique 547 
impulse response in transfer functions 
impulse sequence 486, 515 
in-phase quadrature components 563 
indefinite quadratic forms 49 
independent events 907, 929 
independent variable 259 
inductors 382 
inequality-constrained optimization 844 
inequality constraints in Lagrange 
multipliers 874 
infinite real integrals 331-3 
infinite sequence 483 
initial-value theorem 
of Laplace transforms 
of z transforms 493 
inner (scalar) product 
input-output block diagram 428 
instantaneous source 821 
integral solutions to partial differential 
equations 815-23 
separated solutions 815-17 
singular solutions 817-20 
integral transforms 346 
integrals, Laplace transforms of 371-2 
integration 


436-7 


437-8 


of Fourier series 595-7 
in vector calculus 214-47 
double integrals 219-24 


Gauss’s divergence theorem 241—4 
Green’s theorem 224-9 

line integral 215-18 

Stokes’ theorem 244—7 

surface integrals 230-6 

volume integrals 237—40 


integro-differential equation 371 
inter-arrival time 975 
interval and test 
for correlation 935-6 
for proportion 922-4 
interval estimate 912-13 
inverse Laplace transform 364-5 
evaluation of 365-7 
and first shift theorem 367-9 
inverse Laplace transform operator 364 
inverse mapping 261 
with respect to the circle 271 
of complex functions 268—73 
inverse matrix 5-6 
properties 6 
inverse Nyquist approach 469 
inverse polar plot 469 
inverse transform 364 
inverse z transform operator 494 
inverse z transformation 494 
inverse z transforms 494-501 
techniques 495-501 
inversion of complex functions 268-73 
irrotational motion 209, 246 
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Jacobi methods 35 
Jacobian 227 
Jacobian matrix 884 
Jaeger, J.C. 823 
Jervis, B.W. 717 
joint density function 927 
joint distributions 925-9, 926 
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and marginal distributions 926-8 
Jong, M.T. 715 
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series at 599-601 
Jury, E.I. 520 
Jury stability criterion 520-2 


K 


kernel of Laplace transform 348 
Kirchhoff’s laws 87, 382 
Kraniauskas, P. 527 

Kuhn, 874 
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Lagrange interpolation formula 879 
Lagrange multipliers 870—4, 871 
equality constraints 870—4, 871 
inequality constraints 874 
Laplace equation in partial differential 
equations 725, 731-3 
solution of 785-801 
numerical solution 794-801 
separated solutions 785—92 
Laplace transform 348—69 
bending of beams 424—7 
definition and notation 348-50, 392-5 
derivative of 360-1 
on differential equations 370-80 
ordinary differential equations 372-7 
simultaneous differential 
equations 378-80 
step and impulse functions 403-7 
transforms of derivatives 370-1 
transforms of integrals 371-2 
existence of 353-5 
and Fourier transform 658-60 
frequency response (application) 462-9 
heaviside step function 392-5 
and impulse functions 418—23 
impulse functions 413-14, 415-18 
and heaviside step function 418—23 
inverse transform 364-5 
evaluation of 365-7 
and first shift theorem 358, 367-9 
kernel of 348 
limits 348-50 
as non-anticipatory system 349 
one-sided (unilateral) transform 349 
periodic functions 407-11 
pole placement (application) 470-1 
properties of 355-63 
second shift theorem 397—400 
inversion 400-3 
sifting property 414-15 
simple functions 350-3 
solution to wave equation 756—9 
state-space equations, solution of 450-61 
table of 363 
transfer functions 428-49 
and arbitrary inputs 446-9 
convolution 443—6 
definitions 428-31 
final-value theorem 439-40 
impulse response 436-7 
initial-value theorem 437-8 
stability in 431-6 
two-sided (bilateral) transform 349 
unit step function 392, 395-7 
and z transforms 529-30 
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Laplace transform method 346 
for solution of heat-conduction/diffusion 
equation 772-7 
Laplace transform operator 348 
Laplace transform pairs 348 
table of 363 
Laplacian operator 211 
Laurent series 303-7, 314 
leading diagonal 3 
leading principle minor of matrices 50 
least squares in hill climbing 892-5 
left inverse matrix 77 
left singular vector matrix 71 
Levy 765 
Lewis, P.E. 802 
likelihood ratio 989 
lilinear transform 547 
limit-cycle behaviour 632 
Lindley, D.V. 990 
line integral 215-18, 317 
line spectra 615 
linear dependence of vector spaces 
linear equations of matrices 7-9 
linear mappings of complex functions 261-8 
linear operator 
in Fourier transforms 652 
in Laplace transforms 356 
on z transforms 489 
linear programming 847-69 
simplex algorithm 849-59 
two-phase method 861-9 
linear regression 940 
linear time-variant system 372 
linearity property 
in Fourier series expansion 577-9 
of Fourier transforms 652 
of z transforms 489-90 
local truncation errors on differential 
equations 134-6 
LR methods in matrices 36 
Lyapunov stability analysis 
(application) 104-6 


11-12 
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Maclaurin series expansion 300 
magnification 263, 264 
main lobe 716 
main lobe width 716 
MAPLE 
on differential equations 121, 122, 126, 
139, 144, 149-51, 154, 159, 174 
on Fourier series 572-3 
on Fourier transforms 
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on linear progamming 856, 866 

on matrices 6-7, 9, 10, 21-2, 39, 


80-1 

on partial differentiation equations 733, 
738, 739, 774 

on vector calculus 185, 187, 190-1, 193, 
206, 209 


on z transforms 487, 488, 497, 505, 517 
mapping 259 
in complex differentiation 290-4 
determinants of 274 
polynomial mapping 280-2 
marginal density function 927 
marginal distributions 926-8 
marginally stable linear system 431-2 
marginally stable system 519 


mass 386 

MATLAB 
on differential equations 121, 149-51 
on Fourier series 572-3 


on Fourier transforms 
671-2, 703-5 
on hill climbing 876—9, 882, 886, 890-1, 
893, 895 
on Laplace transforms 353, 359-60, 
361-2, 365-6, 376, 377, 380, 395— 
6, 397, 400, 401, 416, 417, 437, 
459-61 
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on partial differentiation equations 739, 
764, 767, 781, 784, 798, 806, 807, 
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on z transforms 486, 488, 496—7, 498, 
500-1, 505, 517, 542-3 
matrices 2-114 
eigenvalues 14-30 
characteristic equation 
method of Faddeev 16 
and eigenvectors 17-22 
repeated 23-7 
useful properties 27-9 
functions 54-64 
matrix algebra 2-10 
adjoint matrix 5 
basic operations 3-4 
definitions 3 
determinants 3 
inverse matrix 5-6 
properties 6 
linear equations 7-9 
rank 9-10 


647-8, 666, 669, 


15-17 


numerical methods 30-7 
Gerschgorin circles 36-7 
power method 30-6 

reduction to canonical form 39-53 
diagonal form 39-42 
Jordan canonical form 42-5 
quadratic forms 47-53 

singular value decomposition 66-81 


pseudo inverse 75-81 
SVD 72-5 
singular values 68—72 


solution of state equation 89-102 
canonical representation 98-102 
direct form 89-91 


spectral representation of response 95-8 
transition matrix 91 
evaluating 92-4 
state-space representation 82-8 
multi-input-multi-output (MIMO) 
systems 87-8 
single-input-single-output (SISO) 
systems 82-6 
symmetric 29-30 
vector spaces 10-14 
linear dependence 10-12 
transformation between bases 12-13 


matrix 3 
maximum of objective function 853 
mean 907 
when variance unknown 918-20 
mean square error in Fourier series 627 
means, difference between 921-2 
mechanical vibrations (application) 386—90 
memoryless property 976 
meromorphic poles 309 
method of separation of variables 751 
Middleton, R.M. 548, 553 
minimal form 457 
modal form in matrics 96 
modal matrix 39 
modes in matrics 96 
modulation 
in Fourier transforms 655 
moment generating functions 953-7 
definition and applications 953-4 
Poisson approximation to the 


binomial 955—7 
Moore-Penrose pseudo inverse square matrix 
76 


Morse, P.M. 830 
motion in a viscous fluid (application) 
116-17 
moving-average control charts 971-2 
multi-input-multi-output (MIMO) systems 
in Laplace transforms 455-61 
in matrices 87-8 
multi-step methods on differential 
equations 128-34, 131 
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multiple service channels queues 982-3 
multiplication-by-t property 360 
multiplication of matrices 4 
multiplication theorem in Fourier 

series 612-13 


Murdoch, J. 969, 971 


N 


negative-definite quadratic forms 49, 51 
negative-semidefinite quadratic forms 
49, 51 
net circulation integral 218 
Neumann conditions in partial differentiation 
equations 826, 828, 829 
Newton method 884, 885 
Newton-Raphson methods 884 
Newton’s law 386 
Nichols diagram 469 
nodes 761 
non-anticipatory systems 349 
non-basic variables 849, 854 
non-binding constraints 848 
non-conservative force 218 
non-negative eigenvalues 68 
non-square matrix 66 
non-trivial solutions of matrices 8 
nonlinear regression 943-4 
normal distribution 910-11 
normal residuals in regression 941-2 
normalizing eigenvectors 20 
nth harmonic 562 
null matrix 3 
Nyquist approach 469 
Nyquist interval 688 
Nyquist-Shannon sampling theorem 688, 
692 


O 


observable state of matrix 100 

odd functions in Fourier series 
expansion 573-7 

odd periodic extension 590 

offsets 440 

Ohm’s law 382 

one-dimensional heat equation 729 

one-sided Laplace transform 349 

open boundary 826 

Oppenheim, A.V. 717 
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optimization 
chemical processing plant 
(application) 896-8 
heating fin (application) 898—900 
hill climbing 867, 875-95 
Lagrange multipliers 870—4 
linear programming 847-69 
order of pole 308 
order of the system 429, 510 
ordinary differential equations, Laplace 
transforms of 372-7 
orthogonal functions 624-9 
orthogonal matrix 13 
orthogonal set 624 
orthogonality relations 563 
orthonormal set 625 
oscillating systems (application) 603-6 
oscillations of a pendulum 
(application) 170-4 
over determined matrix 75, 78 


P 


Page, E. 983 
parabolic equations 825 
parameters 909 
estimating 912-24 
confidence interval for mean 914-17, 
915 
difference between means 921-2 
distribution of sample average 913-14 
hypothesis tests 912-13 
interval and test for proportion 922—4 
interval estimate 912-13 
mean when variance unknown 918-20 
testing simple hypotheses 917-18 
parasitic solutions in differential equations 
132 
Parseval’s theorem 612, 614, 664 
partial correlation 933 
partial derivative 185 
partial differential equations 724 
arbitrary functions and first-order 
equations 735-40 
boundary conditions 826—30 
finite elements 802-14 
formal classification of 824-6 
heat-conduction or diffusion 
equation 725, 728-31 
solution of 768-84 
Laplace transform method 772-7 
numerical solution 779-84 
separation method 768-72 
sources and sinks for 820-3 
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integral solutions 


separated solutions 
singular solutions 


815-23 


815-17 
817-20 


Laplace equation 725, 731-3 
solution of 785-801 
numerical solution 794-801 
separated solutions 
Poisson equation 734 
Reynolds number 
Schrédinger equation 734 
wave equation 725-8 
solution of 742-67 
D’Alembert solution 742-51, 745 
Laplace transform solution 756—9 
numerical solution 761—7 
separated solutions 


particular integral 
particular integral in Laplace transform 
methods 


Paterson, Colin 548 
path of line integral 


pendulum, oscillations of (application) 


170-4 


period 561 


733 


90 


373 


215 


periodic extension 588 


periodic functions 


phase angle 
phase plane 


562 
83 


561-2 


785-92 


751-6 


phase quadrature components 563 
phase shift 464 
phase spectrum 615, 648, 711 


phases in linear programming 862-6 
point at infinity 303 
Poisson approximation to the binomial 


955-7 
Poisson distribution 909 
Poisson equation 734 
Poisson process in queues 
polar plot 469 
pole placement (application) 470-1 


pole-zero plot 
poles 308, 5 


and eigenvalues 


429 
10 


470 


975-7 


polynomial approximation 879 
polynomial mapping 280-2 
population 912 
population mean 913 


positive constant in matrices 


104 


positive definite function 104 


positive-definite quadratic forms 


positive-semidefinite quadratic forms 


posterior odds 


posterior probabilities 


Powell 888 
power 665 


989 


989 


power method on matrices 


power series 


295-9 


30-6, 32 


49, 51 


49, 51 


power signals 668 
power spectrum 621-3, 622 
practical signal 641 
predictor-corrector methods on differential 
equations 136-41, 138 

principal diagonal 3 
principal part of Laurent series 304 
principle minor of matrices 50 
principle of superposition 446 
prior odds 989 
prior probabilities 989 
probability density function 908 
probability theory 906-12 

Bernoulli distribution 909 

binomial distribution 909 

central limit theorem 910 

proof 956-7 

normal distribution 910-11 

Poisson distribution 909 

random variables 907-9 

tules 907 

sample measures 911-12 
product of eigenvalues 27 
product rule 907 
proportion, interval and test for 922-4 
pseudo inverse square matrix 75-81, 76 
punctured disc 305 
pure resonance 389 


Q 


q (shift) operator 547 
QR methods in matrices 36 
quadratic forms of matrices 47-53, 105 
quadratic polynomial 879 
quasi-Newton method 884 
queues 974-85 
multiple service channels queue 982-3 
Poisson process in 975—7 
problems 974-5 
simulation 983-5 
single service channel queue 978-82 
quiescent state 428 
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Rade, L. 643 

radius of convergence 296 
random variables 907-9 
range charts 973 
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rank correlation 936-7 
rank of a matrix 9-10 
rate of arrival 976 
real integrals 333-4 
real vector space 10 
realization problem 457 
reciprocal basis vectors of matrix 95 
rectangular matrix 66 
rectangular window 712 
regression 938-44, 939 
and correlation 943 
least squares method 939-41 
linear 940 
nonlinear 943-4 
normal residuals 941-2 
regression coefficients 940 
regular function 283 
regular point of f(z) 308, 309 
removable singularity 309 
repeated eigenvalues 23-7 
residual of equation 165 
residue theorem 328-31, 329 
residues 311-16 
resistors 382 
resonance 389, 604 
response in differential equations 372 
Reynolds number in partial differential 
equations 733 
Richardson extrapolation 136 
Riemann sphere 303 
right inverse matrix 77 
right singular vector matrix 71 
Roberts, R.A. 418-19 
robust methods 906 
root mean square (RMS) 614 
rotation 263, 264 
rotational motion 209 
Routh-Hurwitz criterion 434 
row rank matrix 66, 67 
row vector 3 
rows 761 
rule of total probability 987 
Runge-Kutta methods on differential 
equations 141—4 
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sample 912 
sample average 912 
distribution of 913-14 
sample correlation 933-5 
sample measures in probability theory 
911-12 
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sample range 973 
sample space 907 
sample variance 912 
sampling 482, 487-8 
sampling function 621 
scalar field 183, 210 
scalar Lyapunov function 104 
scalar point function 182 
derivatives of 199-202 
gradient 199-200 
scatter diagrams 933 
Schafer, R.W. 717 
Schrédinger equation 734 
Schwarzenbach, J. 442 
second shift property of z transforms 491-2 
second shift theorem 397-400 
inversion 400-3 
separated solutions 
in Laplace equation method 785-92 
of partial differential equations 815—17 
to wave equation 751-6 
separation method for solution of 
heat-conduction/diffusion 
equation 768—72 
service channel 975 
service discipline 975 
service time 975 
set of vectors 11 
Shanno 890 
Shewart attribute control charts 964-7 
Shewart variable control charts 967-8 
shooting method in differential 
equations 162-4 
sifting property 414-15 
signals 347 
significance levels 917 
signum function 671 
similarity transform 39 
simple pole 309 
simplex algorithm 849-53, 850 
general theory 853-9 
simplex method 848 
simplification 2 
simulation, queues 983-5 
simultaneous differential equations, Laplace 
transform on 378-80 
sine function 620 
Singer, A. 835 
single-input-single-output (SISO) 
systems 82-6 
in Laplace transforms 450—4 
in matrices 82-6 
single multivariable searches in hill 
climbing 882-7 
single service channel queue 978—82 
single-variable search in hill climbing 
875-81 
singular points 871 
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singular solutions of partial differential 
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66-81, 72-5 
pseudo inverse 75-81 
singular value matrix 68 
singularities 303, 308-11 
sinks in solution of heat-conduction/diffusion 
equation 820-3 
sinusoids, damped 360 
skew symmetric matrix 3 
slack variables 849 
Snell's law 872 
solenoidal vectors 205 
sources in solution of heat-conduction/ 
diffusion equation 820-3 
Spearman rank correlation coefficient 936 
spectral form in matrics 96 
spectral leakage 714 
spectral matrix 39 
spectral pairs in matrics 95 
spectral representation of response of state 
equations 95-8 
springs 386 
square matrix 3, 68 
square non-singular matrix 76 
stability 
in differential equations 132 
in discrete-linear systems 518—24 
in transfer functions 431-6 
stable linear system 431 
standard deviation 908 
standard form of transfer function 466 
standard normal distribution 910 
standard tableau 853 
state equation 83 
state equation, solution of 89-102 
canonical representation 98-102 
direct form 89-91 
spectral representation of response 95-8 
transition matrix 91 
evaluating 92-4 
state feedback 470 
state-space 2, 83 
state-space form 552 
state-space model 84 
state-space representation 
of higher order systems 156—9 
in Laplace transforms 450-61 
multi-input-multi-output (MIMO) 
systems 455-61 
single-input-single-output (SISO) 
systems 450—4 
in matrices 
multi-input-multi-output (MIMO) 
systems 87-8 
single-input-single-output (SISO) 
systems 82-6 
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state vector 83 time-shift property 653-4 
statistical quality control (application) top hat function 393 
964-73 total differential 196 
Cusum control charts 968-71 in vector calculus 196-9 
moving-average control charts 971-2 trace 3 
range charts 973 trajectory 83 
Shewart attribute control charts 964—7 transfer functions 428-49 
Shewart variable control charts 967-8 and arbitrary inputs 446-9 
Statistics 906 convolution 443—6 
steady-state erors 440 definitions 428-31 
steady-state gain 440 final-value theorem 439-40 
Stearns, S.J. 717 impulse response 436-7 
Steele, N.C. 545, 549, 702, 709 initial-value theorem 437-8 
steepest ascent/descent 882 transfer matrix 456 
step functions transformations 192, 259 
in Fourier transforms 663-75 in vector calculus 192-5 
Laplace transforms on 403-7 of vector spaces 12-13 
step-invariant method 540-3, 541 transition matrix 91 
step size in Euler’s method 120 in discrete-time state equations 533 
stiff differential equations 147-9 evaluating 92-4 
stiffness matrix 806 transition property 91 
Stokes’ theorem 244-7, 245 translation 263, 264 
stream function 248 transmission line 836 
streamline in fluid dynamics transposed matrix 3, 4, 68 
(application) 248-9 properties 4 
subdominant eigenvalue 31 Tranter, W.H. 717 
successive over-relaxation (SOR) travelling waves 744 
method 797-8 trial function 165 
sum of eigenvalues 27 triangular window 717 
superposition integral 443 Tucker 874 
superposition principle 446 Tukey, J.W. 638, 693 
surface integrals 230-6 Tustin transform 547, 549 
surplus variable 862 two-dimensional heat equation 731 
Sylvester's conditions 50, 105 two-phase method 861-9 
symmetric matrix 3, 29-30, 68 two phase strategy 864 
symmetry property 655-7, 656 two-sided Laplace transform 349 
system discrete 510 type Lerror 917 
system frequency response 661 type IL error 917 


system input/output 372 
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unbounded region 855 


tableau form 850 uncontrollable modes in matrics 100 
Taylor series 299-302 under determined matrix 75 

Taylor series expansion 300 unilateral Laplace transform 349, 658 
Taylor theorem 884 unit impulse function 413 

testing simple hypotheses 917-18 unit matrix 3 

text statistic 917 unit pulse 486 

thermal diffusivity 251, 729 unit step function 392, 395—7 
thermally isotropic medium 250 unitary matrix 68 

Thomas algorithm 766 unobservable state of matrix 100 
time as variable 346 upper control limit (UCL) in control 
time-differentiation property 652-3 charts 966 
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solution of 742-67 
V D’Alembert solution 742-51 
Laplace transform solution 756-9 
numerical solution 761-7 


variable 964 separated solutions 751-6 

variance 907 wave propagation under moving load 
unknown, mean when 918-20 (application) 831-4 

variational problems 899 ‘weak’ form 804 

vector calculus 181-256 weighting factor 79 
basic concepts 183-91 weighting function in transfer functions 436 
derivatives of scalar point function Westergren, B. 643 
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